Index: head/en_US.ISO8859-1/articles/fonts/article.xml =================================================================== --- head/en_US.ISO8859-1/articles/fonts/article.xml (revision 50804) +++ head/en_US.ISO8859-1/articles/fonts/article.xml (revision 50805) @@ -1,925 +1,916 @@
Fonts and FreeBSDA Tutorial DaveBodenstab
imdave@synet.net
Wed Aug 7, 1996 $FreeBSD$ &tm-attrib.freebsd; &tm-attrib.adobe; &tm-attrib.apple; &tm-attrib.linux; &tm-attrib.microsoft; &tm-attrib.opengroup; &tm-attrib.general; This document contains a description of the various font files that may be used with FreeBSD and the syscons driver, X11, Ghostscript and Groff. Cookbook examples are provided for switching the syscons display to 80x60 mode, and for using type 1 fonts with the above application programs. Introduction There are many sources of fonts available, and one might ask how they might be used with FreeBSD. The answer can be found by carefully searching the documentation for the component that one would like to use. This is very time consuming, so this tutorial is an attempt to provide a shortcut for others who might be interested. Basic terminology There are many different font formats and associated font file suffixes. A few that will be addressed here are: .pfa, .pfb &postscript; type 1 fonts. The .pfa is the Ascii form and .pfb the Binary form. .afm The font metrics associated with a type 1 font. .pfm The printer font metrics associated with a type 1 font. .ttf A &truetype; font .fot An indirect reference to a TrueType font (not an actual font) .fon, .fnt Bitmapped screen fonts The .fot file is used by &windows; as sort of a symbolic link to the actual &truetype; font (.ttf) file. The .fon font files are also used by Windows. I know of no way to use this font format with FreeBSD. What font formats can I use? Which font file format is useful depends on the application being used. FreeBSD by itself uses no fonts. Application programs and/or drivers may make use of the font files. Here is a small cross reference of application/driver to the font type suffixes: Driver vt .hex syscons .fnt Application Ghostscript .pfa, .pfb, .ttf X11 .pfa, .pfb Groff .pfa, .afm Povray .ttf The .fnt suffix is used quite frequently. I suspect that whenever someone wanted to create a specialized font file for their application, more often than not they chose this suffix. Therefore, it is likely that files with this suffix are not all the same format; specifically, the .fnt files used by syscons under FreeBSD may not be the same format as a .fnt file one encounters in the &ms-dos;/&windows; environment. I have not made any attempt at using other .fnt files other than those provided with FreeBSD. Setting a virtual console to 80x60 line mode First, an 8x8 font must be loaded. To do this, /etc/rc.conf should contain the line (change the font name to an appropriate one for your locale): font8x8="iso-8x8" # font 8x8 from /usr/share/syscons/fonts/* (or NO). The command to actually switch the mode is &man.vidcontrol.1;: &prompt.user; vidcontrol VGA_80x60 Various screen-oriented programs, such as &man.vi.1;, must be able to determine the current screen dimensions. As this is achieved this through ioctl calls to the console driver (such as &man.syscons.4;) they will correctly determine the new screen dimensions. To make this more seamless, one can embed these commands in the startup scripts so it takes place when the system boots. To do this is add this line to /etc/rc.conf allscreens_flags="VGA_80x60" # Set this vidcontrol mode for all virtual screens References: &man.rc.conf.5;, &man.vidcontrol.1;. Using type 1 fonts with <application>X11</application> X11 can use either the .pfa or the .pfb format fonts. The X11 fonts are located in various subdirectories under /usr/X11R6/lib/X11/fonts. Each font file is cross referenced to its X11 name by the contents of the fonts.dir file in each directory. There is already a directory named Type1. The most straight forward way to add a new font is to put it into this directory. A better way is to keep all new fonts in a separate directory and use a symbolic link to the additional font. This allows one to more easily keep track of ones fonts without confusing them with the fonts that were originally provided. For example: Create a directory to contain the font files &prompt.user; mkdir -p /usr/local/share/fonts/type1 &prompt.user; cd /usr/local/share/fonts/type1 Place the .pfa, .pfb and .afm files here One might want to keep readme files, and other documentation for the fonts here also &prompt.user; cp /cdrom/fonts/atm/showboat/showboat.pfb . &prompt.user; cp /cdrom/fonts/atm/showboat/showboat.afm . Maintain an index to cross reference the fonts &prompt.user; echo showboat - InfoMagic CICA, Dec 1994, /fonts/atm/showboat >>INDEX Now, to use a new font with X11, one must make the font file available and update the font name files. The X11 font names look like: - -bitstream-charter-medium-r-normal-xxx-0-0-0-0-p-0-iso8859-1 + -bitstream-charter-medium-r-normal-xxx-0-0-0-0-p-0-iso8859-1 | | | | | | | | | | | | \ \ | | | | | \ \ \ \ \ \ \ +----+- character set | | | | \ \ \ \ \ \ \ +- average width | | | | \ \ \ \ \ \ +- spacing | | | \ \ \ \ \ \ +- vertical res. | | | \ \ \ \ \ +- horizontal res. | | | \ \ \ \ +- points | | | \ \ \ +- pixels | | | \ \ \ - foundry family weight slant width additional style + foundry family weight slant width additional style A new name needs to be created for each new font. If you have some information from the documentation that accompanied the font, then it could serve as the basis for creating the name. If there is no information, then you can get some idea by using &man.strings.1; on the font file. For example: &prompt.user; strings showboat.pfb | more %!FontType1-1.0: Showboat 001.001 %%CreationDate: 1/15/91 5:16:03 PM %%VMusage: 1024 45747 % Generated by Fontographer 3.1 % Showboat 1991 by David Rakowski. Alle Rechte Vorbehalten. FontDirectory/Showboat known{/Showboat findfont dup/UniqueID known{dup /UniqueID get 4962377 eq exch/FontType get 1 eq and}{pop false}ifelse {save true}{false}ifelse}{false}ifelse 12 dict begin /FontInfo 9 dict dup begin /version (001.001) readonly def /FullName (Showboat) readonly def /FamilyName (Showboat) readonly def /Weight (Medium) readonly def /ItalicAngle 0 def /isFixedPitch false def /UnderlinePosition -106 def /UnderlineThickness 16 def /Notice (Showboat 1991 by David Rakowski. Alle Rechte Vorbehalten.) readonly def end readonly def /FontName /Showboat def --stdin-- Using this information, a possible name might be: -type1-Showboat-medium-r-normal-decorative-0-0-0-0-p-0-iso8859-1 The components of our name are: Foundry Lets just name all the new fonts type1. Family The name of the font. Weight Normal, bold, medium, semibold, etc. From the &man.strings.1; output above, it appears that this font has a weight of medium. Slant roman, italic, oblique, etc. Since the ItalicAngle is zero, roman will be used. Width Normal, wide, condensed, extended, etc. Until it can be examined, the assumption will be normal. Additional style Usually omitted, but this will indicate that the font contains decorative capital letters. Spacing proportional or monospaced. Proportional is used since isFixedPitch is false. All of these names are arbitrary, but one should strive to be compatible with the existing conventions. A font is referenced by name with possible wild cards by an X11 program, so the name chosen should make some sense. One might begin by simply using - …-normal-r-normal-…-p-… - + …-normal-r-normal-…-p-… as the name, and then use &man.xfontsel.1; to examine it and adjust the name based on the appearance of the font. So, to complete our example: Make the font accessible to X11 &prompt.user; cd /usr/X11R6/lib/X11/fonts/Type1 &prompt.user; ln -s /usr/local/share/fonts/type1/showboat.pfb . Edit fonts.dir and fonts.scale, adding the line describing the font and incrementing the number of fonts which is found on the first line. &prompt.user; ex fonts.dir :1p 25 :1c 26 . :$a showboat.pfb -type1-showboat-medium-r-normal-decorative-0-0-0-0-p-0-iso8859-1 . :wq fonts.scale seems to be identical to fonts.dir… &prompt.user; cp fonts.dir fonts.scale Tell X11 that things have changed &prompt.user; xset fp rehash Examine the new font &prompt.user; xfontsel -pattern -type1-* References: &man.xfontsel.1;, &man.xset.1;, The X Windows System in a Nutshell, O'Reilly & Associates. Using type 1 fonts with Ghostscript Ghostscript references a font via its Fontmap file. This must be modified in a similar way to the X11 fonts.dir file. Ghostscript can use either the .pfa or the .pfb format fonts. Using the font from the previous example, here is how to use it with Ghostscript: Put the font in Ghostscript's font directory &prompt.user; cd /usr/local/share/ghostscript/fonts &prompt.user; ln -s /usr/local/share/fonts/type1/showboat.pfb . Edit Fontmap so Ghostscript knows about the font &prompt.user; cd /usr/local/share/ghostscript/4.01 &prompt.user; ex Fontmap :$a /Showboat (showboat.pfb) ; % From CICA /fonts/atm/showboat . :wq Use Ghostscript to examine the font &prompt.user; gs prfont.ps Aladdin Ghostscript 4.01 (1996-7-10) Copyright (C) 1996 Aladdin Enterprises, Menlo Park, CA. All rights reserved. This software comes with NO WARRANTY: see the file PUBLIC for details. Loading Times-Roman font from /usr/local/share/ghostscript/fonts/tir_____.pfb... /1899520 581354 1300084 13826 0 done. GS>Showboat DoFont Loading Showboat font from /usr/local/share/ghostscript/fonts/showboat.pfb... 1939688 565415 1300084 16901 0 done. >>showpage, press <return> to continue<< >>showpage, press <return> to continue<< >>showpage, press <return> to continue<< GS>quit References: fonts.txt in the Ghostscript 4.01 distribution Using type 1 fonts with Groff Now that the new font can be used by both X11 and Ghostscript, how can one use the new font with groff? First of all, since we are dealing with type 1 &postscript; fonts, the groff device that is applicable is the ps device. A font file must be created for each font that groff can use. A groff font name is just a file in /usr/share/groff_font/devps. With our example, the font file could be /usr/share/groff_font/devps/SHOWBOAT. The file must be created using tools provided by groff. The first tool is afmtodit. This is not normally installed, so it must be retrieved from the source distribution. I found I had to change the first line of the file, so I did: &prompt.user; cp /usr/src/gnu/usr.bin/groff/afmtodit/afmtodit.pl /tmp &prompt.user; ex /tmp/afmtodit.pl :1c #!/usr/bin/perl -P- . :wq This tool will create the groff font file from the metrics file (.afm suffix.) Continuing with our example: Many .afm files are in Mac format… ^M delimited lines We need to convert them to &unix; style ^J delimited lines &prompt.user; cd /tmp &prompt.user; cat /usr/local/share/fonts/type1/showboat.afm | tr '\015' '\012' >showboat.afm Now create the groff font file &prompt.user; cd /usr/share/groff_font/devps &prompt.user; /tmp/afmtodit.pl -d DESC -e text.enc /tmp/showboat.afm generate/textmap SHOWBOAT The font can now be referenced with the name SHOWBOAT. If Ghostscript is used to drive the printers on the system, then nothing more needs to be done. However, if true &postscript; printers are used, then the font must be downloaded to the printer in order for the font to be used (unless the printer happens to have the showboat font built in or on an accessible font disk.) The final step is to create a downloadable font. The pfbtops tool is used to create the .pfa format of the font, and the download file is modified to reference the new font. The download file must reference the internal name of the font. This can easily be determined from the groff font file as illustrated: Create the .pfa font file &prompt.user; pfbtops /usr/local/share/fonts/type1/showboat.pfb >showboat.pfa Of course, if the .pfa file is already available, just use a symbolic link to reference it. Get the internal font name &prompt.user; fgrep internalname SHOWBOAT internalname Showboat Tell groff that the font must be downloaded &prompt.user; ex download :$a Showboat showboat.pfa . :wq To test the font: &prompt.user; cd /tmp &prompt.user; cat >example.t <<EOF .sp 5 .ps 16 This is an example of the Showboat font: .br .ps 48 .vs (\n(.s+2)p .sp .ft SHOWBOAT ABCDEFGHI .br JKLMNOPQR .br STUVWXYZ .sp .ps 16 .vs (\n(.s+2)p .fp 5 SHOWBOAT .ft R To use it for the first letter of a paragraph, it will look like: .sp 50p \s(48\f5H\s0\fRere is the first sentence of a paragraph that uses the showboat font as its first letter. Additional vertical space must be used to allow room for the larger letter. EOF &prompt.user; groff -Tps example.t >example.ps To use ghostscript/ghostview &prompt.user; ghostview example.ps To print it &prompt.user; lpr -Ppostscript example.ps References: /usr/src/gnu/usr.bin/groff/afmtodit/afmtodit.man, &man.groff.font.5;, &man.groff.char.7;, &man.pfbtops.1;. Converting TrueType fonts to a groff/PostScript format for groff This potentially requires a bit of work, simply because it depends on some utilities that are not installed as part of the base system. They are: ttf2pf TrueType to PostScript conversion utilities. This allows conversion of a TrueType font to an ascii font metric (.afm) file. Currently available at http://sunsite.icm.edu.pl/pub/GUST/contrib/BachoTeX98/ttf2pf/. Note: These files are PostScript programs and must be downloaded to disk by holding down the Shift key when clicking on the link. Otherwise, your browser may try to launch ghostview to view them. The files of interest are: GS_TTF.PS PF2AFM.PS ttf2pf.ps The funny upper/lower case is due to their being intended also for DOS shells. ttf2pf.ps makes use of the others as upper case, so any renaming must be consistent with this. (Actually, GS_TTF.PS and PFS2AFM.PS are supposedly part of the Ghostscript distribution, but it is just as easy to use these as an isolated utility. FreeBSD does not seem to include the latter.) You also may want to have these installed to /usr/local/share/groff_font/devps(?). afmtodit Creates font files for use with groff from ascii font metrics file. This usually resides in the directory, /usr/src/contrib/groff/afmtodit, and requires some work to get going. If you are paranoid about working in the /usr/src tree, simply copy the contents of the above directory to a work location. In the work area, you will need to make the utility. Just type: - # make -f Makefile.sub afmtodit - + # make -f Makefile.sub afmtodit You may also need to copy /usr/contrib/groff/devps/generate/textmap to /usr/share/groff_font/devps/generate if it does not already exist. Once all these utilities are in place, you are ready to commence: Create the .afm file by typing: - % gs -dNODISPLAY -q -- ttf2pf.ps TTF_name PS_font_name AFM_name - + % gs -dNODISPLAY -q -- ttf2pf.ps TTF_name PS_font_name AFM_name Where, TTF_name is your TrueType font file, PS_font_name is the file name for the .pfa file, AFM_name is the name you wish for the .afm file. If you do not specify output file names for the .pfa or .afm files, then default names will be generated from the TrueType font file name. This also produces a .pfa file, the ascii PostScript font metrics file (.pfb is for the binary form). This will not be needed, but could (I think) be useful for a fontserver. For example, to convert the 30f9 Barcode font using the default file names, use the following command: % gs -dNODISPLAY -- ttf2pf.ps 3of9.ttf Aladdin Ghostscript 5.10 (1997-11-23) Copyright (C) 1997 Aladdin Enterprises, Menlo Park, CA. All rights reserved. This software comes with NO WARRANTY: see the file PUBLIC for details. -Converting 3of9.ttf to 3of9.pfa and 3of9.afm. - +Converting 3of9.ttf to 3of9.pfa and 3of9.afm. If you want the converted fonts to be stored in A.pfa and B.afm, then use this command: % gs -dNODISPLAY -- ttf2pf.ps 3of9.ttf A B Aladdin Ghostscript 5.10 (1997-11-23) Copyright (C) 1997 Aladdin Enterprises, Menlo Park, CA. All rights reserved. This software comes with NO WARRANTY: see the file PUBLIC for details. -Converting 3of9.ttf to A.pfa and B.afm. - +Converting 3of9.ttf to A.pfa and B.afm. Create the groff PostScript file: Change directories to /usr/share/groff_font/devps so as to make the following command easier to execute. You will probably need root privileges for this. (Or, if you are paranoid about working there, make sure you reference the files DESC, text.enc and generate/textmap as being in this directory.) - % afmtodit -d DESC -e text.enc file.afm \ - generate/textmap PS_font_name - + % afmtodit -d DESC -e text.enc file.afm generate/textmap PS_font_name Where, file.afm is the AFM_name created by ttf2pf.ps above, and PS_font_name is the font name used from that command, as well as the name that &man.groff.1; will use for references to this font. For example, assuming you used the first tiff2pf.ps command above, then the 3of9 Barcode font can be created using the command: - % afmtodit -d DESC -e text.enc 3of9.afm \ - generate/textmap 3of9 - - + % afmtodit -d DESC -e text.enc 3of9.afm generate/textmap 3of9 + Ensure that the resulting PS_font_name file (e.g., 3of9 in the example above) is located in the directory /usr/share/groff_font/devps by copying or moving it there. Note that if ttf2pf.ps assigns a font name using the one it finds in the TrueType font file and you want to use a different name, you must edit the .afm file prior to running afmtodit. This name must also match the one used in the Fontmap file if you wish to pipe &man.groff.1; into &man.gs.1;. Can TrueType fonts be used with other programs? The TrueType font format is used by Windows, Windows 95, and Mac's. It is quite popular and there are a great number of fonts available in this format. Unfortunately, there are few applications that I am aware of that can use this format: Ghostscript and Povray come to mind. Ghostscript's support, according to the documentation, is rudimentary and the results are likely to be inferior to type 1 fonts. Povray version 3 also has the ability to use TrueType fonts, but I rather doubt many people will be creating documents as a series of raytraced pages :-). This rather dismal situation may soon change. The FreeType Project is currently developing a useful set of FreeType tools: The xfsft font server for X11 can serve TrueType fonts in addition to regular fonts. Though currently in beta, it is said to be quite usable. See Juliusz Chroboczek's page for further information. Porting instructions for FreeBSD can be found at Stephen Montgomery's software page. xfstt is another font server for X11, available under ftp://sunsite.unc.edu/pub/Linux/X11/fonts/. A program called ttf2bdf can produce BDF files suitable for use in an X environment from TrueType files. Linux binaries are said to be available from ftp://crl.nmsu.edu/CLR/multiling/General/. and others … Where can additional fonts be obtained? Many fonts are available on the Internet. They are either entirely free, or are share-ware. In addition many fonts are available in the x11-fonts/ category in the ports collection Additional questions What use are the .pfm files? Can one generate the .afm file from a .pfa or .pfb? How to generate the groff character mapping files for PostScript fonts with non-standard character names? Can xditview and devX?? devices be set up to access all the new fonts? It would be good to have examples of using TrueType fonts with Povray and Ghostscript.
Index: head/en_US.ISO8859-1/articles/freebsd-update-server/article.xml =================================================================== --- head/en_US.ISO8859-1/articles/freebsd-update-server/article.xml (revision 50804) +++ head/en_US.ISO8859-1/articles/freebsd-update-server/article.xml (revision 50805) @@ -1,835 +1,835 @@ FreeBSD Update Server"> ]>
Build Your Own &os; Update Server Jason Helfman
&a.jgh.email;
 2009 2010 2011 2013 Jason Helfman &tm-attrib.freebsd; &tm-attrib.general; &tm-attrib.intel; &tm-attrib.amd; $FreeBSD$ $FreeBSD$ This article describes building an internal &fbus.ap;. The freebsd-update-server is written by &a.cperciva.email;, Security Officer Emeritus of &os;. For users that think it is convenient to update their systems against an official update server, building their own &fbus.ap; may help to extend its functionality by supporting manually-tweaked &os; releases or by providing a local mirror that will allow faster updates for a number of machines. Acknowledgments This article was subsequently printed at BSD Magazine. Introduction Experienced users or administrators are often responsible for several machines or environments. They understand the difficult demands and challenges of maintaining such an infrastructure. Running a &fbus.ap; makes it easier to deploy security and software patches to selected test machines before rolling them out to production. It also means a number of systems can be updated from the local network rather than a potentially slower Internet connection. This article outlines the steps involved in creating an internal &fbus.ap;. Prerequisites To build an internal &fbus.ap; some requirements should be met. A running &os; system. At a minimum, updates require building on a &os; release greater than or equal to the target release version for distribution. A user account with at least 4 GB of available space. This will allow the creation of updates for 7.1 and 7.2, but the exact space requirements may change from version to version. An &man.ssh.1; account on a remote machine to upload distributed updates. A web server, like Apache, with over half of the space required for the build. For instance, test builds for 7.1 and 7.2 consume a total amount of 4 GB, and the webserver space needed to distribute these updates is 2.6 GB. Basic knowledge of shell scripting with Bourne shell, &man.sh.1;. Configuration: Installation & Setup Download the freebsd-update-server software by installing devel/subversion, and execute: &prompt.user; svn co http://svn.freebsd.org/base/user/cperciva/freebsd-update-build freebsd-update-server Update scripts/build.conf appropriately. It is sourced during all build operations. Here is the default build.conf, which should be modified to suit your environment. # Main configuration file for FreeBSD Update builds. The # release-specific configuration data is lower down in # the scripts tree. # Location from which to fetch releases export FTP=ftp://ftp2.freebsd.org/pub/FreeBSD/releases # Host platform export HOSTPLATFORM=`uname -m` # Host name to use inside jails export BUILDHOSTNAME=${HOSTPLATFORM}-builder.daemonology.net # Location of SSH key export SSHKEY=/root/.ssh/id_dsa # SSH account into which files are uploaded MASTERACCT=builder@wadham.daemonology.net # Directory into which files are uploaded MASTERDIR=update-master.freebsd.org Parameters for consideration would be: This is the location where ISO images are downloaded from (by the fetchiso() subroutine of scripts/build.subr). The location configured is not limited to FTP URIs. Any URI scheme supported by standard &man.fetch.1; utility should work fine. Customizations to the fetchiso() code can be installed by copying the default build.subr script to the release and architecture-specific area at scripts/RELEASE/ARCHITECTURE/build.subr and applying local changes. The name of the build host. This information will be displayed on updated systems when issuing: &prompt.user; uname -v The SSH key for uploading files to the update server. A key pair can be created by typing ssh-keygen -t dsa. This parameter is optional; standard password authentication will be used as a fallback authentication method when SSHKEY is not defined. The &man.ssh-keygen.1; manual page has more detailed information about SSH and the appropriate steps for creating and using one. Account for uploading files to the update server. Directory on the update server where files are uploaded to. The default build.conf shipped with the freebsd-update-server sources is suitable for building &arch.i386; releases of &os;. As an example of building an update server for other architectures, the following steps outline the configuration changes needed for &arch.amd64;: Create a build environment for &arch.amd64;: &prompt.user; mkdir -p /usr/local/freebsd-update-server/scripts/7.2-RELEASE/amd64 Install a build.conf in the newly created build directory. The build configuration options for &os; 7.2-RELEASE on &arch.amd64; should be similar to: # SHA256 hash of RELEASE disc1.iso image. export RELH=1ea1f6f652d7c5f5eab7ef9f8edbed50cb664b08ed761850f95f48e86cc71ef5 # Components of the world, source, and kernels export WORLDPARTS="base catpages dict doc games info manpages proflibs lib32" export SOURCEPARTS="base bin contrib crypto etc games gnu include krb5 \ lib libexec release rescue sbin secure share sys tools \ ubin usbin cddl" export KERNELPARTS="generic" # EOL date export EOL=1275289200 The &man.sha256.1; hash key for the desired release, is published within the respective release announcement. To generate the "End of Life" number for build.conf, refer to the "Estimated EOL" posted on the &os; Security Website. The value of EOL can be derived from the date listed on the web site, using the &man.date.1; utility, for example: &prompt.user; date -j -f '%Y%m%d-%H%M%S' '20090401-000000' '+%s' Building Update Code The first step is to run scripts/make.sh. This will build some binaries, create directories, and generate an RSA signing key used for approving builds. In this step, a passphrase will have to be supplied for the final creation of the signing key. &prompt.root; sh scripts/make.sh cc -O2 -fno-strict-aliasing -pipe findstamps.c -o findstamps findstamps.c: In function 'usage': findstamps.c:45: warning: incompatible implicit declaration of built-in function 'exit' cc -O2 -fno-strict-aliasing -pipe unstamp.c -o unstamp install findstamps ../bin install unstamp ../bin rm -f findstamps unstamp Generating RSA private key, 4096 bit long modulus ................................................................................++ ...................++ e is 65537 (0x10001) Public key fingerprint: 27ef53e48dc869eea6c3136091cc6ab8589f967559824779e855d58a2294de9e Encrypting signing key for root enter aes-256-cbc encryption password: Verifying - enter aes-256-cbc encryption password: Keep a note of the generated key fingerprint. This value is required in /etc/freebsd-update.conf for binary updates. At this point, we are ready to stage a build. &prompt.root; cd /usr/local/freebsd-update-server &prompt.root; sh scripts/init.sh amd64 7.2-RELEASE What follows is a sample of an initial build run. &prompt.root; sh scripts/init.sh amd64 7.2-RELEASE Mon Aug 24 16:04:36 PDT 2009 Starting fetch for FreeBSD/amd64 7.2-RELEASE /usr/local/freebsd-update-server/work/7.2-RELE100% of 588 MB 359 kBps 00m00s Mon Aug 24 16:32:38 PDT 2009 Verifying disc1 hash for FreeBSD/amd64 7.2-RELEASE Mon Aug 24 16:32:44 PDT 2009 Extracting components for FreeBSD/amd64 7.2-RELEASE Mon Aug 24 16:34:05 PDT 2009 Constructing world+src image for FreeBSD/amd64 7.2-RELEASE Mon Aug 24 16:35:57 PDT 2009 Extracting world+src for FreeBSD/amd64 7.2-RELEASE Mon Aug 24 23:36:24 UTC 2009 Building world for FreeBSD/amd64 7.2-RELEASE Tue Aug 25 00:31:29 UTC 2009 Distributing world for FreeBSD/amd64 7.2-RELEASE Tue Aug 25 00:32:36 UTC 2009 Building and distributing kernels for FreeBSD/amd64 7.2-RELEASE Tue Aug 25 00:44:44 UTC 2009 Constructing world components for FreeBSD/amd64 7.2-RELEASE Tue Aug 25 00:44:56 UTC 2009 Distributing source for FreeBSD/amd64 7.2-RELEASE Mon Aug 24 17:46:18 PDT 2009 Moving components into staging area for FreeBSD/amd64 7.2-RELEASE Mon Aug 24 17:46:33 PDT 2009 Identifying extra documentation for FreeBSD/amd64 7.2-RELEASE Mon Aug 24 17:47:13 PDT 2009 Extracting extra docs for FreeBSD/amd64 7.2-RELEASE Mon Aug 24 17:47:18 PDT 2009 Indexing release for FreeBSD/amd64 7.2-RELEASE Mon Aug 24 17:50:44 PDT 2009 Indexing world0 for FreeBSD/amd64 7.2-RELEASE Files built but not released: Files released but not built: Files which differ by more than contents: Files which differ between release and build: kernel|generic|/GENERIC/hptrr.ko kernel|generic|/GENERIC/kernel src|sys|/sys/conf/newvers.sh world|base|/boot/loader world|base|/boot/pxeboot world|base|/etc/mail/freebsd.cf world|base|/etc/mail/freebsd.submit.cf world|base|/etc/mail/sendmail.cf world|base|/etc/mail/submit.cf world|base|/lib/libcrypto.so.5 world|base|/usr/bin/ntpq world|base|/usr/lib/libalias.a world|base|/usr/lib/libalias_cuseeme.a world|base|/usr/lib/libalias_dummy.a world|base|/usr/lib/libalias_ftp.a ... Then the build of the world is performed again, with world patches. A more detailed explanation may be found in scripts/build.subr. During this second build cycle, the network time protocol daemon, &man.ntpd.8;, is turned off. Per &a.cperciva.email;, Security Officer Emeritus of &os;, "the freebsd-update-server build code needs to identify timestamps which are stored in files so that they can be ignored when comparing builds to determine which files need to be updated. This timestamp-finding works by doing two builds 400 days apart and comparing the results." Mon Aug 24 17:54:07 PDT 2009 Extracting world+src for FreeBSD/amd64 7.2-RELEASE Wed Sep 29 00:54:34 UTC 2010 Building world for FreeBSD/amd64 7.2-RELEASE Wed Sep 29 01:49:42 UTC 2010 Distributing world for FreeBSD/amd64 7.2-RELEASE Wed Sep 29 01:50:50 UTC 2010 Building and distributing kernels for FreeBSD/amd64 7.2-RELEASE Wed Sep 29 02:02:56 UTC 2010 Constructing world components for FreeBSD/amd64 7.2-RELEASE Wed Sep 29 02:03:08 UTC 2010 Distributing source for FreeBSD/amd64 7.2-RELEASE Tue Sep 28 19:04:31 PDT 2010 Moving components into staging area for FreeBSD/amd64 7.2-RELEASE Mon Aug 24 19:04:46 PDT 2009 Extracting extra docs for FreeBSD/amd64 7.2-RELEASE Mon Aug 24 19:04:51 PDT 2009 Indexing world1 for FreeBSD/amd64 7.2-RELEASE Mon Aug 24 19:08:04 PDT 2009 Locating build stamps for FreeBSD/amd64 7.2-RELEASE Mon Aug 24 19:10:19 PDT 2009 Cleaning staging area for FreeBSD/amd64 7.2-RELEASE Mon Aug 24 19:10:19 PDT 2009 Preparing to copy files into staging area for FreeBSD/amd64 7.2-RELEASE Mon Aug 24 19:10:20 PDT 2009 Copying data files into staging area for FreeBSD/amd64 7.2-RELEASE Mon Aug 24 12:16:57 PDT 2009 Copying metadata files into staging area for FreeBSD/amd64 7.2-RELEASE Mon Aug 24 12:16:59 PDT 2009 Constructing metadata index and tag for FreeBSD/amd64 7.2-RELEASE Files found which include build stamps: kernel|generic|/GENERIC/hptrr.ko kernel|generic|/GENERIC/kernel world|base|/boot/loader world|base|/boot/pxeboot world|base|/etc/mail/freebsd.cf world|base|/etc/mail/freebsd.submit.cf world|base|/etc/mail/sendmail.cf world|base|/etc/mail/submit.cf world|base|/lib/libcrypto.so.5 world|base|/usr/bin/ntpq world|base|/usr/include/osreldate.h world|base|/usr/lib/libalias.a world|base|/usr/lib/libalias_cuseeme.a world|base|/usr/lib/libalias_dummy.a world|base|/usr/lib/libalias_ftp.a ... Finally, the build completes. Values of build stamps, excluding library archive headers: v1.2 (Aug 25 2009 00:40:36) v1.2 (Aug 25 2009 00:38:22) @(#)FreeBSD 7.2-RELEASE #0: Tue Aug 25 00:38:29 UTC 2009 FreeBSD 7.2-RELEASE #0: Tue Aug 25 00:38:29 UTC 2009 root@server.myhost.com:/usr/obj/usr/src/sys/GENERIC 7.2-RELEASE Mon Aug 24 23:55:25 UTC 2009 Mon Aug 24 23:55:25 UTC 2009 ##### built by root@server.myhost.com on Tue Aug 25 00:16:15 UTC 2009 ##### built by root@server.myhost.com on Tue Aug 25 00:16:15 UTC 2009 ##### built by root@server.myhost.com on Tue Aug 25 00:16:15 UTC 2009 ##### built by root@server.myhost.com on Tue Aug 25 00:16:15 UTC 2009 Mon Aug 24 23:46:47 UTC 2009 ntpq 4.2.4p5-a Mon Aug 24 23:55:53 UTC 2009 (1) * Copyright (c) 1992-2009 The FreeBSD Project. Mon Aug 24 23:46:47 UTC 2009 Mon Aug 24 23:55:40 UTC 2009 Aug 25 2009 ntpd 4.2.4p5-a Mon Aug 24 23:55:52 UTC 2009 (1) ntpdate 4.2.4p5-a Mon Aug 24 23:55:53 UTC 2009 (1) ntpdc 4.2.4p5-a Mon Aug 24 23:55:53 UTC 2009 (1) Tue Aug 25 00:21:21 UTC 2009 Tue Aug 25 00:21:21 UTC 2009 Tue Aug 25 00:21:21 UTC 2009 Mon Aug 24 23:46:47 UTC 2009 FreeBSD/amd64 7.2-RELEASE initialization build complete. Please review the list of build stamps printed above to confirm that they look sensible, then run # sh -e approve.sh amd64 7.2-RELEASE to sign the release. Approve the build if everything is correct. More information on determining this can be found in the distributed source file named USAGE. Execute scripts/approve.sh, as directed. This will sign the release, and move components into a staging area suitable for uploading. &prompt.root; cd /usr/local/freebsd-update-server &prompt.root; sh scripts/mountkey.sh &prompt.root; sh -e scripts/approve.sh amd64 7.2-RELEASE Wed Aug 26 12:50:06 PDT 2009 Signing build for FreeBSD/amd64 7.2-RELEASE Wed Aug 26 12:50:06 PDT 2009 Copying files to patch source directories for FreeBSD/amd64 7.2-RELEASE Wed Aug 26 12:50:06 PDT 2009 Copying files to upload staging area for FreeBSD/amd64 7.2-RELEASE Wed Aug 26 12:50:07 PDT 2009 Updating databases for FreeBSD/amd64 7.2-RELEASE Wed Aug 26 12:50:07 PDT 2009 Cleaning staging area for FreeBSD/amd64 7.2-RELEASE After the approval process is complete, the upload procedure may be started. &prompt.root; cd /usr/local/freebsd-update-server &prompt.root; sh scripts/upload.sh amd64 7.2-RELEASE In the event update code needs to be re-uploaded, this may be done by changing to the public distributions directory for the target release and updating attributes of the uploaded file. &prompt.root; cd /usr/local/freebsd-update-server/pub/7.2-RELEASE/amd64 &prompt.root; touch -t 200801010101.01 uploaded The uploaded files will need to be in the document root of the webserver in order for updates to be distributed. The exact configuration will vary depending on the web server used. For the Apache web server, please refer to the Configuration of Apache servers section in the Handbook. Update client's KeyPrint and ServerName in /etc/freebsd-update.conf, and perform updates as instructed in the &os; Update section of the Handbook. In order for &fbus.ap; to work properly, updates for both the current release and the release one wants to upgrade to need to be built. This is necessary for determining the differences of files between releases. For example, when upgrading a &os; system from 7.1-RELEASE to 7.2-RELEASE, updates will need to be built and uploaded to your distribution server for both versions. For reference, the entire run of init.sh is attached. Building a Patch Every time a security advisory or security notice is announced, a patch update can be built. For this example, 7.1-RELEASE will be used. A couple of assumptions are made for a different release build: Setup the correct directory structure for the initial build. Perform an initial build for 7.1-RELEASE. Create the patch directory of the respective release under /usr/local/freebsd-update-server/patches/. &prompt.user; mkdir -p /usr/local/freebsd-update-server/patches/7.1-RELEASE/ &prompt.user; cd /usr/local/freebsd-update-server/patches/7.1-RELEASE As an example, take the patch for &man.named.8;. Read the advisory, and grab the necessary file from &os; Security Advisories. More information on interpreting the advisory, can be found in the &os; Handbook. In the security brief, this advisory is called SA-09:12.bind. After downloading the file, it is required to rename the file to an appropriate patch level. It is suggested to keep this consistent with official &os; patch levels, but its name may be freely chosen. For this build, let us follow the currently established practice of &os; and call this p7. Rename the file: &prompt.user; cd /usr/local/freebsd-update-server/patches/7.1-RELEASE/; mv bind.patch 7-SA-09:12.bind When running a patch level build, it is assumed that previous patches are in place. When a patch build is run, it will run all patches contained in the patch directory. There can be custom patches added to any build. Use the number zero, or any other number. It is up to the administrator of the &fbus.ap; to take appropriate measures to verify the authenticity of every patch. At this point, a diff is ready to be built. The software checks first to see if a scripts/init.sh has been run on the respective release prior to running the diff build. &prompt.root; cd /usr/local/freebsd-update-server &prompt.root; sh scripts/diff.sh amd64 7.1-RELEASE 7 What follows is a sample of a differential build run. &prompt.root; sh -e scripts/diff.sh amd64 7.1-RELEASE 7 Wed Aug 26 10:09:59 PDT 2009 Extracting world+src for FreeBSD/amd64 7.1-RELEASE-p7 Wed Aug 26 17:10:25 UTC 2009 Building world for FreeBSD/amd64 7.1-RELEASE-p7 Wed Aug 26 18:05:11 UTC 2009 Distributing world for FreeBSD/amd64 7.1-RELEASE-p7 Wed Aug 26 18:06:16 UTC 2009 Building and distributing kernels for FreeBSD/amd64 7.1-RELEASE-p7 Wed Aug 26 18:17:50 UTC 2009 Constructing world components for FreeBSD/amd64 7.1-RELEASE-p7 Wed Aug 26 18:18:02 UTC 2009 Distributing source for FreeBSD/amd64 7.1-RELEASE-p7 Wed Aug 26 11:19:23 PDT 2009 Moving components into staging area for FreeBSD/amd64 7.1-RELEASE-p7 Wed Aug 26 11:19:37 PDT 2009 Extracting extra docs for FreeBSD/amd64 7.1-RELEASE-p7 Wed Aug 26 11:19:42 PDT 2009 Indexing world0 for FreeBSD/amd64 7.1-RELEASE-p7 Wed Aug 26 11:23:02 PDT 2009 Extracting world+src for FreeBSD/amd64 7.1-RELEASE-p7 Thu Sep 30 18:23:29 UTC 2010 Building world for FreeBSD/amd64 7.1-RELEASE-p7 Thu Sep 30 19:18:15 UTC 2010 Distributing world for FreeBSD/amd64 7.1-RELEASE-p7 Thu Sep 30 19:19:18 UTC 2010 Building and distributing kernels for FreeBSD/amd64 7.1-RELEASE-p7 Thu Sep 30 19:30:52 UTC 2010 Constructing world components for FreeBSD/amd64 7.1-RELEASE-p7 Thu Sep 30 19:31:03 UTC 2010 Distributing source for FreeBSD/amd64 7.1-RELEASE-p7 Thu Sep 30 12:32:25 PDT 2010 Moving components into staging area for FreeBSD/amd64 7.1-RELEASE-p7 Wed Aug 26 12:32:39 PDT 2009 Extracting extra docs for FreeBSD/amd64 7.1-RELEASE-p7 Wed Aug 26 12:32:43 PDT 2009 Indexing world1 for FreeBSD/amd64 7.1-RELEASE-p7 Wed Aug 26 12:35:54 PDT 2009 Locating build stamps for FreeBSD/amd64 7.1-RELEASE-p7 Wed Aug 26 12:36:58 PDT 2009 Reverting changes due to build stamps for FreeBSD/amd64 7.1-RELEASE-p7 Wed Aug 26 12:37:14 PDT 2009 Cleaning staging area for FreeBSD/amd64 7.1-RELEASE-p7 Wed Aug 26 12:37:14 PDT 2009 Preparing to copy files into staging area for FreeBSD/amd64 7.1-RELEASE-p7 Wed Aug 26 12:37:15 PDT 2009 Copying data files into staging area for FreeBSD/amd64 7.1-RELEASE-p7 Wed Aug 26 12:43:23 PDT 2009 Copying metadata files into staging area for FreeBSD/amd64 7.1-RELEASE-p7 Wed Aug 26 12:43:25 PDT 2009 Constructing metadata index and tag for FreeBSD/amd64 7.1-RELEASE-p7 ... Files found which include build stamps: kernel|generic|/GENERIC/hptrr.ko kernel|generic|/GENERIC/kernel world|base|/boot/loader world|base|/boot/pxeboot world|base|/etc/mail/freebsd.cf world|base|/etc/mail/freebsd.submit.cf world|base|/etc/mail/sendmail.cf world|base|/etc/mail/submit.cf world|base|/lib/libcrypto.so.5 world|base|/usr/bin/ntpq world|base|/usr/include/osreldate.h world|base|/usr/lib/libalias.a world|base|/usr/lib/libalias_cuseeme.a world|base|/usr/lib/libalias_dummy.a world|base|/usr/lib/libalias_ftp.a ... Values of build stamps, excluding library archive headers: v1.2 (Aug 26 2009 18:13:46) v1.2 (Aug 26 2009 18:11:44) @(#)FreeBSD 7.1-RELEASE-p7 #0: Wed Aug 26 18:11:50 UTC 2009 FreeBSD 7.1-RELEASE-p7 #0: Wed Aug 26 18:11:50 UTC 2009 root@server.myhost.com:/usr/obj/usr/src/sys/GENERIC 7.1-RELEASE-p7 Wed Aug 26 17:29:15 UTC 2009 Wed Aug 26 17:29:15 UTC 2009 ##### built by root@server.myhost.com on Wed Aug 26 17:49:58 UTC 2009 ##### built by root@server.myhost.com on Wed Aug 26 17:49:58 UTC 2009 ##### built by root@server.myhost.com on Wed Aug 26 17:49:58 UTC 2009 ##### built by root@server.myhost.com on Wed Aug 26 17:49:58 UTC 2009 Wed Aug 26 17:20:39 UTC 2009 ntpq 4.2.4p5-a Wed Aug 26 17:29:42 UTC 2009 (1) * Copyright (c) 1992-2009 The FreeBSD Project. Wed Aug 26 17:20:39 UTC 2009 Wed Aug 26 17:29:30 UTC 2009 Aug 26 2009 ntpd 4.2.4p5-a Wed Aug 26 17:29:41 UTC 2009 (1) ntpdate 4.2.4p5-a Wed Aug 26 17:29:42 UTC 2009 (1) ntpdc 4.2.4p5-a Wed Aug 26 17:29:42 UTC 2009 (1) Wed Aug 26 17:55:02 UTC 2009 Wed Aug 26 17:55:02 UTC 2009 Wed Aug 26 17:55:02 UTC 2009 Wed Aug 26 17:20:39 UTC 2009 ... Updates are printed, and approval is requested. New updates: kernel|generic|/GENERIC/kernel.symbols|f|0|0|0555|0|7c8dc176763f96ced0a57fc04e7c1b8d793f27e006dd13e0b499e1474ac47e10| kernel|generic|/GENERIC/kernel|f|0|0|0555|0|33197e8cf15bbbac263d17f39c153c9d489348c2c534f7ca1120a1183dec67b1| kernel|generic|/|d|0|0|0755|0|| src|base|/|d|0|0|0755|0|| src|bin|/|d|0|0|0755|0|| src|cddl|/|d|0|0|0755|0|| src|contrib|/contrib/bind9/bin/named/update.c|f|0|10000|0644|0|4d434abf0983df9bc47435670d307fa882ef4b348ed8ca90928d250f42ea0757| src|contrib|/contrib/bind9/lib/dns/openssldsa_link.c|f|0|10000|0644|0|c6805c39f3da2a06dd3f163f26c314a4692d4cd9a2d929c0acc88d736324f550| src|contrib|/contrib/bind9/lib/dns/opensslrsa_link.c|f|0|10000|0644|0|fa0f7417ee9da42cc8d0fd96ad24e7a34125e05b5ae075bd6e3238f1c022a712| ... FreeBSD/amd64 7.1-RELEASE update build complete. Please review the list of build stamps printed above and the list of updated files to confirm that they look sensible, then run # sh -e approve.sh amd64 7.1-RELEASE to sign the build. Follow the same process as noted before for approving a build: &prompt.root; sh -e scripts/approve.sh amd64 7.1-RELEASE Wed Aug 26 12:50:06 PDT 2009 Signing build for FreeBSD/amd64 7.1-RELEASE Wed Aug 26 12:50:06 PDT 2009 Copying files to patch source directories for FreeBSD/amd64 7.1-RELEASE Wed Aug 26 12:50:06 PDT 2009 Copying files to upload staging area for FreeBSD/amd64 7.1-RELEASE Wed Aug 26 12:50:07 PDT 2009 Updating databases for FreeBSD/amd64 7.1-RELEASE Wed Aug 26 12:50:07 PDT 2009 Cleaning staging area for FreeBSD/amd64 7.1-RELEASE The FreeBSD/amd64 7.1-RELEASE update build has been signed and is ready to be uploaded. Remember to run # sh -e umountkey.sh to unmount the decrypted key once you have finished signing all the new builds. After approving the build, upload the software: &prompt.root; cd /usr/local/freebsd-update-server &prompt.root; sh scripts/upload.sh amd64 7.1-RELEASE For reference, the entire run of diff.sh is attached. Tips If a custom release is built using the native make release procedure, freebsd-update-server code will work from your release. As an example, a release without ports or documentation can be built by clearing functionality pertaining to documentation subroutines findextradocs (), addextradocs () and altering the download location in fetchiso (), respectively, in scripts/build.subr. As a last step, change the &man.sha256.1; hash in build.conf under your respective release and architecture and you are ready to build off your custom release. - # Compare ${WORKDIR}/release and ${WORKDIR}/$1, identify which parts - # of the world|doc subcomponent are missing from the latter, and - # build a tarball out of them. - findextradocs () { - } + # Compare ${WORKDIR}/release and ${WORKDIR}/$1, identify which parts +# of the world|doc subcomponent are missing from the latter, and +# build a tarball out of them. +findextradocs () { +} - # Add extra docs to ${WORKDIR}/$1 - addextradocs () { - } +# Add extra docs to ${WORKDIR}/$1 +addextradocs () { +} Adding flags to buildworld and obj targets in the scripts/build.subr script may speed up processing depending on the hardware used, however it is not necessary. Using these flags in other targets is not recommended, as it may cause the build to become unreliable. - # Build the world + # Build the world log "Building world" cd /usr/src && make -j 2 ${COMPATFLAGS} buildworld 2>&1 # Distribute the world log "Distributing world" cd /usr/src/release && make -j 2 obj && - make ${COMPATFLAGS} release.1 release.2 2>&1 + make ${COMPATFLAGS} release.1 release.2 2>&1 Create an appropriate DNS SRV record for the update server, and put others behind it with variable weights. Using this facility will provide update mirrors, however this tip is not necessary unless you wish to provide a redundant service. - _http._tcp.update.myserver.com. IN SRV 0 2 80 host1.myserver.com. - SRV 0 1 80 host2.myserver.com. - SRV 0 0 80 host3.myserver.com. + _http._tcp.update.myserver.com. IN SRV 0 2 80 host1.myserver.com. + IN SRV 0 1 80 host2.myserver.com. + IN SRV 0 0 80 host3.myserver.com.
Index: head/en_US.ISO8859-1/articles/hubs/article.xml =================================================================== --- head/en_US.ISO8859-1/articles/hubs/article.xml (revision 50804) +++ head/en_US.ISO8859-1/articles/hubs/article.xml (revision 50805) @@ -1,696 +1,694 @@
Mirroring FreeBSD JunKuriyama
kuriyama@FreeBSD.org
 ValentinoVaschetto
logo@FreeBSD.org
 DanielLang
dl@leo.org
 KenSmith
kensmith@FreeBSD.org
 &tm-attrib.freebsd; &tm-attrib.general; $FreeBSD$ $FreeBSD$ An in-progress article on how to mirror FreeBSD, aimed at hub administrators. We are not accepting new mirrors at this time. Contact Information The Mirror System Coordinators can be reached through email at mirror-admin@FreeBSD.org. There is also a &a.hubs;. Requirements for FreeBSD mirrors Disk Space Disk space is one of the most important requirements. Depending on the set of releases, architectures, and degree of completeness you want to mirror, a huge amount of disk space may be consumed. Also keep in mind that official mirrors are probably required to be complete. The web pages should always be mirrored completely. Also note that the numbers stated here are reflecting the current state (at &rel2.current;-RELEASE/&rel.current;-RELEASE). Further development and releases will only increase the required amount. Also make sure to keep some (ca. 10-20%) extra space around just to be sure. Here are some approximate figures: Full FTP Distribution: 1.4 TB CTM deltas: 10 GB Web pages: 1GB The current disk usage of FTP Distribution can be found at ftp://ftp.FreeBSD.org/pub/FreeBSD/dir.sizes. Network Connection/Bandwidth Of course, you need to be connected to the Internet. The required bandwidth depends on your intended use of the mirror. If you just want to mirror some parts of FreeBSD for local use at your site/intranet, the demand may be much smaller than if you want to make the files publicly available. If you intend to become an official mirror, the bandwidth required will be even higher. We can only give rough estimates here: Local site, no public access: basically no minimum, but < 2 Mbps could make syncing too slow. Unofficial public site: 34 Mbps is probably a good start. Official site: > 100 Mbps is recommended, and your host should be connected as close as possible to your border router. System Requirements, CPU, RAM One thing this depends on the expected number of clients, which is determined by the server's policy. It is also affected by the types of services you want to offer. Plain FTP or HTTP services may not require a huge amount of resources. Watch out if you provide rsync. This can have a huge impact on CPU and memory requirements as it is considered a memory hog. The following are just examples to give you a very rough hint. For a moderately visited site that offers rsync, you might consider a current CPU with around 800MHz - 1 GHz, and at least 512MB RAM. This is probably the minimum you want for an official site. For a frequently used site you definitely need more RAM (consider 2GB as a good start) and possibly more CPU, which could also mean that you need to go for a SMP system. You also want to consider a fast disk subsystem. Operations on the SVN repository require a fast disk subsystem (RAID is highly advised). A SCSI controller that has a cache of its own can also speed up things since most of these services incur a large number of small modifications to the disk. Services to offer Every mirror site is required to have a set of core services available. In addition to these required services, there are a number of optional services that server administrators may choose to offer. This section explains which services you can provide and how to go about implementing them. FTP (required for FTP fileset) This is one of the most basic services, and it is required for each mirror offering public FTP distributions. FTP access must be anonymous, and no upload/download ratios are allowed (a ridiculous thing anyway). Upload capability is not required (and must never be allowed for the FreeBSD file space). Also the FreeBSD archive should be available under the path /pub/FreeBSD. There is a lot of software available which can be set up to allow anonymous FTP (in alphabetical order). /usr/libexec/ftpd: FreeBSD's own ftpd can be used. Be sure to read &man.ftpd.8;. ftp/ncftpd: A commercial package, free for educational use. ftp/oftpd: An ftpd designed with security as a main focus. ftp/proftpd: A modular and very flexible ftpd. ftp/pure-ftpd: Another ftpd developed with security in mind. ftp/twoftpd: As above. ftp/vsftpd: The very secure ftpd. FreeBSD's ftpd, proftpd and maybe ncftpd are among the most commonly used FTPds. The others do not have a large userbase among mirror sites. One thing to consider is that you may need flexibility in limiting how many simultaneous connections are allowed, thus limiting how much network bandwidth and system resources are consumed. Rsync (optional for FTP fileset) Rsync is often offered for access to the contents of the FTP area of FreeBSD, so other mirror sites can use your system as their source. The protocol is different from FTP in many ways. It is much more bandwidth friendly, as only differences between files are transferred instead of whole files when they change. Rsync does require a significant amount of memory for each instance. The size depends on the size of the synced module in terms of the number of directories and files. Rsync can use rsh and ssh (now default) as a transport, or use its own protocol for stand-alone access (this is the preferred method for public rsync servers). Authentication, connection limits, and other restrictions may be applied. There is just one software package available: net/rsync HTTP (required for web pages, optional for FTP fileset) If you want to offer the FreeBSD web pages, you will need to install a web server. You may optionally offer the FTP fileset via HTTP. The choice of web server software is left up to the mirror administrator. Some of the most popular choices are: www/apache22: Apache is the most widely deployed web server on the Internet. It is used extensively by the FreeBSD Project. www/thttpd: If you are going to be serving a large amount of static content you may find that using an application such as thttpd is more efficient than Apache. It is optimized for excellent performance on FreeBSD. www/boa: Boa is another alternative to thttpd and Apache. It should provide considerably better performance than Apache for purely static content. It does not, at the time of this writing, contain the same set of optimizations for FreeBSD that are found in thttpd. www/nginx: Nginx is a high performance edge web server with a low memory footprint and key features to build a modern and efficient web infrastructure. Features include a HTTP server, HTTP and mail reverse proxy, caching, load balancing, compression, request throttling, connection multiplexing and reuse, SSL offload and HTTP media streaming. How to Mirror FreeBSD Ok, now you know the requirements and how to offer the services, but not how to get it. :-) This section explains how to actually mirror the various parts of FreeBSD, what tools to use, and where to mirror from. Mirroring the FTP site The FTP area is the largest amount of data that needs to be mirrored. It includes the distribution sets required for network installation, the branches which are actually snapshots of checked-out source trees, the ISO Images to write CD-ROMs with the installation distribution, a live file system, and a snapshot of the ports tree. All of course for various FreeBSD versions, and various architectures. The best way to mirror the FTP area is rsync. You can install the port net/rsync and then use rsync to sync with your upstream host. rsync is already mentioned in . Since rsync access is not required, your preferred upstream site may not allow it. You may need to hunt around a little bit to find a site that allows rsync access. Since the number of rsync clients will have a significant impact on the server machine, most admins impose limitations on their server. For a mirror, you should ask the site maintainer you are syncing from about their policy, and maybe an exception for your host (since you are a mirror). A command line to mirror FreeBSD might look like: - &prompt.user; rsync -vaHz --delete rsync://ftp4.de.FreeBSD.org/FreeBSD/ /pub/FreeBSD/ - + &prompt.user; rsync -vaHz --delete rsync://ftp4.de.FreeBSD.org/FreeBSD/ /pub/FreeBSD/ Consult the documentation for rsync, which is also available at http://rsync.samba.org/, about the various options to be used with rsync. If you sync the whole module (unlike subdirectories), be aware that the module-directory (here "FreeBSD") will not be created, so you cannot omit the target directory. Also you might want to set up a script framework that calls such a command via &man.cron.8;. Mirroring the WWW pages The FreeBSD website should only be mirrored via rsync. A command line to mirror the FreeBSD web site might look like: - &prompt.user; rsync -vaHz --delete rsync://bit0.us-west.freebsd.org/FreeBSD-www-data/ /usr/local/www/ - + &prompt.user; rsync -vaHz --delete rsync://bit0.us-west.freebsd.org/FreeBSD-www-data/ /usr/local/www/ Mirroring Packages Due to very high requirements of bandwidth, storage and adminstration the &os; Project has decided not to allow public mirrors of packages. For sites with lots of machines, it might be advantagous to run a caching HTTP proxy for the &man.pkg.8; process. Alternatively specific packages and their dependencies can be fetched by running something like the following: &prompt.user; pkg fetch -d -o /usr/local/mirror vim Once those packages have been fetched, the repository metadata must be generated by running: &prompt.user; pkg repo /usr/local/mirror Once the packages have been fetched and the metadata for the repository has been generated, serve the packages up to the client machines via HTTP. For additional information see the man pages for &man.pkg.8;, specifically the &man.pkg-repo.8; page. How often should I mirror? Every mirror should be updated at a minimum of once per day. Certainly a script with locking to prevent multiple runs happening at the same time will be needed to run from &man.cron.8;. Since nearly every admin does this in their own way, specific instructions cannot be provided. It could work something like this: Put the command to run your mirroring application in a script. Use of a plain /bin/sh script is recommended. Add some output redirections so diagnostic messages are logged to a file. Test if your script works. Check the logs. Use &man.crontab.1; to add the script to the appropriate user's &man.crontab.5;. This should be a different user than what your FTP daemon runs as so that if file permissions inside your FTP area are not world-readable those files can not be accessed by anonymous FTP. This is used to stage releases — making sure all of the official mirror sites have all of the necessary release files on release day. Here are some recommended schedules: FTP fileset: daily WWW pages: daily Where to mirror from This is an important issue. So this section will spend some effort to explain the backgrounds. We will say this several times: under no circumstances should you mirror from ftp.FreeBSD.org. A few words about the organization Mirrors are organized by country. All official mirrors have a DNS entry of the form ftpN.CC.FreeBSD.org. CC (i.e. country code) is the top level domain (TLD) of the country where this mirror is located. N is a number, telling that the host would be the Nth mirror in that country. (Same applies to wwwN.CC.FreeBSD.org, etc.) There are mirrors with no CC part. These are the mirror sites that are very well connected and allow a large number of concurrent users. ftp.FreeBSD.org is actually two machines, one currently located in Denmark and the other in the United States. It is NOT a master site and should never be used to mirror from. Lots of online documentation leads interactiveusers to ftp.FreeBSD.org so automated mirroring systems should find a different machine to mirror from. Additionally there exists a hierarchy of mirrors, which is described in terms of tiers. The master sites are not referred to but can be described as Tier-0. Mirrors that mirror from these sites can be considered Tier-1, mirrors of Tier-1-mirrors, are Tier-2, etc. Official sites are encouraged to be of a low tier, but the lower the tier the higher the requirements in terms as described in . Also access to low-tier-mirrors may be restricted, and access to master sites is definitely restricted. The tier-hierarchy is not reflected by DNS and generally not documented anywhere except for the master sites. However, official mirrors with low numbers like 1-4, are usually Tier-1 (this is just a rough hint, and there is no rule). Ok, but where should I get the stuff now? Under no circumstances should you mirror from ftp.FreeBSD.org. The short answer is: from the site that is closest to you in Internet terms, or gives you the fastest access. I just want to mirror from somewhere! If you have no special intentions or requirements, the statement in applies. This means: Check for those which provide fastest access (number of hops, round-trip-times) and offer the services you intend to use (like rsync). Contact the administrators of your chosen site stating your request, and asking about their terms and policies. Set up your mirror as described above. I am an official mirror, what is the right site for me? In general the description in still applies. Of course you may want to put some weight on the fact that your upstream should be of a low tier. There are some other considerations about official mirrors that are described in . I want to access the master sites! If you have good reasons and good prerequisites, you may want and get access to one of the master sites. Access to these sites is generally restricted, and there are special policies for access. If you are already an official mirror, this certainly helps you getting access. In any other case make sure your country really needs another mirror. If it already has three or more, ask the zone administrator (hostmaster@CC.FreeBSD.org) or &a.hubs; first. Whoever helped you become, an official should have helped you gain access to an appropriate upstream host, either one of the master sites or a suitable Tier-1 site. If not, you can send email to mirror-admin@FreeBSD.org to request help with that. There is one master site for the FTP fileset. ftp-master.FreeBSD.org This is the master site for the FTP fileset. ftp-master.FreeBSD.org provides rsync access, in addition to FTP. Refer to . Mirrors are also encouraged to allow rsync access for the FTP contents, since they are Tier-1-mirrors. Official Mirrors Official mirrors are mirrors that a) have a FreeBSD.org DNS entry (usually a CNAME). b) are listed as an official mirror in the FreeBSD documentation (like handbook). So far to distinguish official mirrors. Official mirrors are not necessarily Tier-1-mirrors. However you probably will not find a Tier-1-mirror, that is not also official. Special Requirements for official (tier-1) mirrors It is not so easy to state requirements for all official mirrors, since the project is sort of tolerant here. It is more easy to say, what official tier-1 mirrors are required to. All other official mirrors can consider this a big should. Tier-1 mirrors are required to: carry the complete fileset allow access to other mirror sites provide FTP and rsync access Furthermore, admins should be subscribed to the &a.hubs;. See this link for details, how to subscribe. It is very important for a hub administrator, especially Tier-1 hub admins, to check the release schedule for the next FreeBSD release. This is important because it will tell you when the next release is scheduled to come out, and thus giving you time to prepare for the big spike of traffic which follows it. It is also important that hub administrators try to keep their mirrors as up-to-date as possible (again, even more crucial for Tier-1 mirrors). If Mirror1 does not update for a while, lower tier mirrors will begin to mirror old data from Mirror1 and thus begins a downward spiral... Keep your mirrors up to date! How to become official then? We are not accepting any new mirrors at this time. Some statistics from mirror sites Here are links to the stat pages of your favorite mirrors (a.k.a. the only ones who feel like providing stats). FTP site statistics ftp.is.FreeBSD.org - hostmaster@is.FreeBSD.org - (Bandwidth) (FTP processes) (HTTP processes) ftp2.ru.FreeBSD.org - mirror@macomnet.ru - (Bandwidth) (HTTP and FTP users)
Index: head/en_US.ISO8859-1/books/developers-handbook/ipv6/chapter.xml =================================================================== --- head/en_US.ISO8859-1/books/developers-handbook/ipv6/chapter.xml (revision 50804) +++ head/en_US.ISO8859-1/books/developers-handbook/ipv6/chapter.xml (revision 50805) @@ -1,1571 +1,1560 @@ IPv6 Internals IPv6/IPsec Implementation YoshinobuInoueContributed by This section should explain IPv6 and IPsec related implementation internals. These functionalities are derived from KAME project IPv6 Conformance The IPv6 related functions conforms, or tries to conform to the latest set of IPv6 specifications. For future reference we list some of the relevant documents below (NOTE: this is not a complete list - this is too hard to maintain...). For details please refer to specific chapter in the document, RFCs, manual pages, or comments in the source code. Conformance tests have been performed on the KAME STABLE kit at TAHI project. Results can be viewed at http://www.tahi.org/report/KAME/. We also attended Univ. of New Hampshire IOL tests (http://www.iol.unh.edu/) in the past, with our past snapshots. RFC1639: FTP Operation Over Big Address Records (FOOBAR) RFC2428 is preferred over RFC1639. FTP clients will first try RFC2428, then RFC1639 if failed. RFC1886: DNS Extensions to support IPv6 RFC1933: Transition Mechanisms for IPv6 Hosts and Routers IPv4 compatible address is not supported. automatic tunneling (described in 4.3 of this RFC) is not supported. &man.gif.4; interface implements IPv[46]-over-IPv[46] tunnel in a generic way, and it covers "configured tunnel" described in the spec. See 23.5.1.5 in this document for details. RFC1981: Path MTU Discovery for IPv6 RFC2080: RIPng for IPv6 usr.sbin/route6d support this. RFC2292: Advanced Sockets API for IPv6 For supported library functions/kernel APIs, see sys/netinet6/ADVAPI. RFC2362: Protocol Independent Multicast-Sparse Mode (PIM-SM) RFC2362 defines packet formats for PIM-SM. draft-ietf-pim-ipv6-01.txt is written based on this. RFC2373: IPv6 Addressing Architecture supports node required addresses, and conforms to the scope requirement. RFC2374: An IPv6 Aggregatable Global Unicast Address Format supports 64-bit length of Interface ID. RFC2375: IPv6 Multicast Address Assignments Userland applications use the well-known addresses assigned in the RFC. RFC2428: FTP Extensions for IPv6 and NATs RFC2428 is preferred over RFC1639. FTP clients will first try RFC2428, then RFC1639 if failed. RFC2460: IPv6 specification RFC2461: Neighbor discovery for IPv6 See 23.5.1.2 in this document for details. RFC2462: IPv6 Stateless Address Autoconfiguration See 23.5.1.4 in this document for details. RFC2463: ICMPv6 for IPv6 specification See 23.5.1.9 in this document for details. RFC2464: Transmission of IPv6 Packets over Ethernet Networks RFC2465: MIB for IPv6: Textual Conventions and General Group Necessary statistics are gathered by the kernel. Actual IPv6 MIB support is provided as a patchkit for ucd-snmp. RFC2466: MIB for IPv6: ICMPv6 group Necessary statistics are gathered by the kernel. Actual IPv6 MIB support is provided as patchkit for ucd-snmp. RFC2467: Transmission of IPv6 Packets over FDDI Networks RFC2497: Transmission of IPv6 packet over ARCnet Networks RFC2553: Basic Socket Interface Extensions for IPv6 IPv4 mapped address (3.7) and special behavior of IPv6 wildcard bind socket (3.8) are supported. See 23.5.1.12 in this document for details. RFC2675: IPv6 Jumbograms See 23.5.1.7 in this document for details. RFC2710: Multicast Listener Discovery for IPv6 RFC2711: IPv6 router alert option draft-ietf-ipngwg-router-renum-08: Router renumbering for IPv6 draft-ietf-ipngwg-icmp-namelookups-02: IPv6 Name Lookups Through ICMP draft-ietf-ipngwg-icmp-name-lookups-03: IPv6 Name Lookups Through ICMP draft-ietf-pim-ipv6-01.txt: PIM for IPv6 &man.pim6dd.8; implements dense mode. &man.pim6sd.8; implements sparse mode. draft-itojun-ipv6-tcp-to-anycast-00: Disconnecting TCP connection toward IPv6 anycast address draft-yamamoto-wideipv6-comm-model-00 See 23.5.1.6 in this document for details. draft-ietf-ipngwg-scopedaddr-format-00.txt : An Extension of Format for IPv6 Scoped Addresses Neighbor Discovery Neighbor Discovery is fairly stable. Currently Address Resolution, Duplicated Address Detection, and Neighbor Unreachability Detection are supported. In the near future we will be adding Proxy Neighbor Advertisement support in the kernel and Unsolicited Neighbor Advertisement transmission command as admin tool. If DAD fails, the address will be marked "duplicated" and message will be generated to syslog (and usually to console). The "duplicated" mark can be checked with &man.ifconfig.8;. It is administrators' responsibility to check for and recover from DAD failures. The behavior should be improved in the near future. Some of the network driver loops multicast packets back to itself, even if instructed not to do so (especially in promiscuous mode). In such cases DAD may fail, because DAD engine sees inbound NS packet (actually from the node itself) and considers it as a sign of duplicate. You may want to look at #if condition marked "heuristics" in sys/netinet6/nd6_nbr.c:nd6_dad_timer() as workaround (note that the code fragment in "heuristics" section is not spec conformant). Neighbor Discovery specification (RFC2461) does not talk about neighbor cache handling in the following cases: when there was no neighbor cache entry, node received unsolicited RS/NS/NA/redirect packet without link-layer address neighbor cache handling on medium without link-layer address (we need a neighbor cache entry for IsRouter bit) For first case, we implemented workaround based on discussions on IETF ipngwg mailing list. For more details, see the comments in the source code and email thread started from (IPng 7155), dated Feb 6 1999. IPv6 on-link determination rule (RFC2461) is quite different from assumptions in BSD network code. At this moment, no on-link determination rule is supported where default router list is empty (RFC2461, section 5.2, last sentence in 2nd paragraph - note that the spec misuse the word "host" and "node" in several places in the section). To avoid possible DoS attacks and infinite loops, only 10 options on ND packet is accepted now. Therefore, if you have 20 prefix options attached to RA, only the first 10 prefixes will be recognized. If this troubles you, please ask it on FREEBSD-CURRENT mailing list and/or modify nd6_maxndopt in sys/netinet6/nd6.c. If there are high demands we may provide sysctl knob for the variable. Scope Index IPv6 uses scoped addresses. Therefore, it is very important to specify scope index (interface index for link-local address, or site index for site-local address) with an IPv6 address. Without scope index, scoped IPv6 address is ambiguous to the kernel, and kernel will not be able to determine the outbound interface for a packet. Ordinary userland applications should use advanced API (RFC2292) to specify scope index, or interface index. For similar purpose, sin6_scope_id member in sockaddr_in6 structure is defined in RFC2553. However, the semantics for sin6_scope_id is rather vague. If you care about portability of your application, we suggest you to use advanced API rather than sin6_scope_id. In the kernel, an interface index for link-local scoped address is embedded into 2nd 16bit-word (3rd and 4th byte) in IPv6 address. For example, you may see something like: - fe80:1::200:f8ff:fe01:6317 - + fe80:1::200:f8ff:fe01:6317 in the routing table and interface address structure (struct in6_ifaddr). The address above is a link-local unicast address which belongs to a network interface whose interface identifier is 1. The embedded index enables us to identify IPv6 link local addresses over multiple interfaces effectively and with only a little code change. Routing daemons and configuration programs, like &man.route6d.8; and &man.ifconfig.8;, will need to manipulate the "embedded" scope index. These programs use routing sockets and ioctls (like SIOCGIFADDR_IN6) and the kernel API will return IPv6 addresses with 2nd 16bit-word filled in. The APIs are for manipulating kernel internal structure. Programs that use these APIs have to be prepared about differences in kernels anyway. When you specify scoped address to the command line, NEVER write the embedded form (such as ff02:1::1 or fe80:2::fedc). This is not supposed to work. Always use standard form, like ff02::1 or fe80::fedc, with command line option for specifying interface (like ping6 -I ne0 ff02::1). In general, if a command does not have command line option to specify outgoing interface, that command is not ready to accept scoped address. This may seem to be opposite from IPv6's premise to support "dentist office" situation. We believe that specifications need some improvements for this. Some of the userland tools support extended numeric IPv6 syntax, as documented in draft-ietf-ipngwg-scopedaddr-format-00.txt. You can specify outgoing link, by using name of the outgoing interface like "fe80::1%ne0". This way you will be able to specify link-local scoped address without much trouble. To use this extension in your program, you will need to use &man.getaddrinfo.3;, and &man.getnameinfo.3; with NI_WITHSCOPEID. The implementation currently assumes 1-to-1 relationship between a link and an interface, which is stronger than what specs say. Plug and Play Most of the IPv6 stateless address autoconfiguration is implemented in the kernel. Neighbor Discovery functions are implemented in the kernel as a whole. Router Advertisement (RA) input for hosts is implemented in the kernel. Router Solicitation (RS) output for endhosts, RS input for routers, and RA output for routers are implemented in the userland. Assignment of link-local, and special addresses IPv6 link-local address is generated from IEEE802 address (Ethernet MAC address). Each of interface is assigned an IPv6 link-local address automatically, when the interface becomes up (IFF_UP). Also, direct route for the link-local address is added to routing table. Here is an output of netstat command: Internet6: Destination Gateway Flags Netif Expire fe80:1::%ed0/64 link#1 UC ed0 fe80:2::%ep0/64 link#2 UC ep0 Interfaces that has no IEEE802 address (pseudo interfaces like tunnel interfaces, or ppp interfaces) will borrow IEEE802 address from other interfaces, such as Ethernet interfaces, whenever possible. If there is no IEEE802 hardware attached, a last resort pseudo-random value, MD5(hostname), will be used as source of link-local address. If it is not suitable for your usage, you will need to configure the link-local address manually. If an interface is not capable of handling IPv6 (such as lack of multicast support), link-local address will not be assigned to that interface. See section 2 for details. Each interface joins the solicited multicast address and the link-local all-nodes multicast addresses (e.g. fe80::1:ff01:6317 and ff02::1, respectively, on the link the interface is attached). In addition to a link-local address, the loopback address (::1) will be assigned to the loopback interface. Also, ::1/128 and ff01::/32 are automatically added to routing table, and loopback interface joins node-local multicast group ff01::1. Stateless address autoconfiguration on hosts In IPv6 specification, nodes are separated into two categories: routers and hosts. Routers forward packets addressed to others, hosts does not forward the packets. net.inet6.ip6.forwarding defines whether this node is router or host (router if it is 1, host if it is 0). When a host hears Router Advertisement from the router, a host may autoconfigure itself by stateless address autoconfiguration. This behavior can be controlled by net.inet6.ip6.accept_rtadv (host autoconfigures itself if it is set to 1). By autoconfiguration, network address prefix for the receiving interface (usually global address prefix) is added. Default route is also configured. Routers periodically generate Router Advertisement packets. To request an adjacent router to generate RA packet, a host can transmit Router Solicitation. To generate a RS packet at any time, use the rtsol command. &man.rtsold.8; daemon is also available. &man.rtsold.8; generates Router Solicitation whenever necessary, and it works great for nomadic usage (notebooks/laptops). If one wishes to ignore Router Advertisements, use sysctl to set net.inet6.ip6.accept_rtadv to 0. To generate Router Advertisement from a router, use the &man.rtadvd.8; daemon. Note that, IPv6 specification assumes the following items, and nonconforming cases are left unspecified: Only hosts will listen to router advertisements Hosts have single network interface (except loopback) Therefore, this is unwise to enable net.inet6.ip6.accept_rtadv on routers, or multi-interface host. A misconfigured node can behave strange (nonconforming configuration allowed for those who would like to do some experiments). To summarize the sysctl knob: accept_rtadv forwarding role of the node --- --- --- 0 0 host (to be manually configured) 0 1 router 1 0 autoconfigured host (spec assumes that host has single interface only, autoconfigured host with multiple interface is out-of-scope) 1 1 invalid, or experimental (out-of-scope of spec) RFC2462 has validation rule against incoming RA prefix information option, in 5.5.3 (e). This is to protect hosts from malicious (or misconfigured) routers that advertise very short prefix lifetime. There was an update from Jim Bound to ipngwg mailing list (look for "(ipng 6712)" in the archive) and it is implemented Jim's update. See 23.5.1.2 in the document for relationship between DAD and autoconfiguration. Generic tunnel interface GIF (Generic InterFace) is a pseudo interface for configured tunnel. Details are described in &man.gif.4;. Currently v6 in v6 v6 in v4 v4 in v6 v4 in v4 are available. Use &man.gifconfig.8; to assign physical (outer) source and destination address to gif interfaces. Configuration that uses same address family for inner and outer IP header (v4 in v4, or v6 in v6) is dangerous. It is very easy to configure interfaces and routing tables to perform infinite level of tunneling. Please be warned. gif can be configured to be ECN-friendly. See 23.5.4.5 for ECN-friendliness of tunnels, and &man.gif.4; for how to configure. If you would like to configure an IPv4-in-IPv6 tunnel with gif interface, read &man.gif.4; carefully. You will need to remove IPv6 link-local address automatically assigned to the gif interface. Source Address Selection Current source selection rule is scope oriented (there are some exceptions - see below). For a given destination, a source IPv6 address is selected by the following rule: If the source address is explicitly specified by the user (e.g. via the advanced API), the specified address is used. If there is an address assigned to the outgoing interface (which is usually determined by looking up the routing table) that has the same scope as the destination address, the address is used. This is the most typical case. If there is no address that satisfies the above condition, choose a global address assigned to one of the interfaces on the sending node. If there is no address that satisfies the above condition, and destination address is site local scope, choose a site local address assigned to one of the interfaces on the sending node. If there is no address that satisfies the above condition, choose the address associated with the routing table entry for the destination. This is the last resort, which may cause scope violation. For instance, ::1 is selected for ff01::1, fe80:1::200:f8ff:fe01:6317 for fe80:1::2a0:24ff:feab:839b (note that embedded interface index - described in 23.5.1.3 - helps us choose the right source address. Those embedded indices will not be on the wire). If the outgoing interface has multiple address for the scope, a source is selected longest match basis (rule 3). Suppose 2001:0DB8:808:1:200:f8ff:fe01:6317 and 2001:0DB8:9:124:200:f8ff:fe01:6317 are given to the outgoing interface. 2001:0DB8:808:1:200:f8ff:fe01:6317 is chosen as the source for the destination 2001:0DB8:800::1. Note that the above rule is not documented in the IPv6 spec. It is considered "up to implementation" item. There are some cases where we do not use the above rule. One example is connected TCP session, and we use the address kept in tcb as the source. Another example is source address for Neighbor Advertisement. Under the spec (RFC2461 7.2.2) NA's source should be the target address of the corresponding NS's target. In this case we follow the spec rather than the above longest-match rule. For new connections (when rule 1 does not apply), deprecated addresses (addresses with preferred lifetime = 0) will not be chosen as source address if other choices are available. If no other choices are available, deprecated address will be used as a last resort. If there are multiple choice of deprecated addresses, the above scope rule will be used to choose from those deprecated addresses. If you would like to prohibit the use of deprecated address for some reason, configure net.inet6.ip6.use_deprecated to 0. The issue related to deprecated address is described in RFC2462 5.5.4 (NOTE: there is some debate underway in IETF ipngwg on how to use "deprecated" address). Jumbo Payload The Jumbo Payload hop-by-hop option is implemented and can be used to send IPv6 packets with payloads longer than 65,535 octets. But currently no physical interface whose MTU is more than 65,535 is supported, so such payloads can be seen only on the loopback interface (i.e. lo0). If you want to try jumbo payloads, you first have to reconfigure the kernel so that the MTU of the loopback interface is more than 65,535 bytes; add the following to the kernel configuration file: options "LARGE_LOMTU" #To test jumbo payload and recompile the new kernel. Then you can test jumbo payloads by the &man.ping6.8; command with -b and -s options. The -b option must be specified to enlarge the size of the socket buffer and the -s option specifies the length of the packet, which should be more than 65,535. For example, type as follows: &prompt.user; ping6 -b 70000 -s 68000 ::1 The IPv6 specification requires that the Jumbo Payload option must not be used in a packet that carries a fragment header. If this condition is broken, an ICMPv6 Parameter Problem message must be sent to the sender. specification is followed, but you cannot usually see an ICMPv6 error caused by this requirement. When an IPv6 packet is received, the frame length is checked and compared to the length specified in the payload length field of the IPv6 header or in the value of the Jumbo Payload option, if any. If the former is shorter than the latter, the packet is discarded and statistics are incremented. You can see the statistics as output of &man.netstat.8; command with `-s -p ip6' option: &prompt.user; netstat -s -p ip6 ip6: (snip) 1 with data size < data length So, kernel does not send an ICMPv6 error unless the erroneous packet is an actual Jumbo Payload, that is, its packet size is more than 65,535 bytes. As described above, currently no physical interface with such a huge MTU is supported, so it rarely returns an ICMPv6 error. TCP/UDP over jumbogram is not supported at this moment. This is because we have no medium (other than loopback) to test this. Contact us if you need this. IPsec does not work on jumbograms. This is due to some specification twists in supporting AH with jumbograms (AH header size influences payload length, and this makes it real hard to authenticate inbound packet with jumbo payload option as well as AH). There are fundamental issues in *BSD support for jumbograms. We would like to address those, but we need more time to finalize these. To name a few: mbuf pkthdr.len field is typed as "int" in 4.4BSD, so it will not hold jumbogram with len > 2G on 32bit architecture CPUs. If we would like to support jumbogram properly, the field must be expanded to hold 4G + IPv6 header + link-layer header. Therefore, it must be expanded to at least int64_t (u_int32_t is NOT enough). We mistakingly use "int" to hold packet length in many places. We need to convert them into larger integral type. It needs a great care, as we may experience overflow during packet length computation. We mistakingly check for ip6_plen field of IPv6 header for packet payload length in various places. We should be checking mbuf pkthdr.len instead. ip6_input() will perform sanity check on jumbo payload option on input, and we can safely use mbuf pkthdr.len afterwards. TCP code needs a careful update in bunch of places, of course. Loop prevention in header processing IPv6 specification allows arbitrary number of extension headers to be placed onto packets. If we implement IPv6 packet processing code in the way BSD IPv4 code is implemented, kernel stack may overflow due to long function call chain. sys/netinet6 code is carefully designed to avoid kernel stack overflow. Because of this, sys/netinet6 code defines its own protocol switch structure, as "struct ip6protosw" (see netinet6/ip6protosw.h). There is no such update to IPv4 part (sys/netinet) for compatibility, but small change is added to its pr_input() prototype. So "struct ipprotosw" is also defined. Because of this, if you receive IPsec-over-IPv4 packet with massive number of IPsec headers, kernel stack may blow up. IPsec-over-IPv6 is okay. (Off-course, for those all IPsec headers to be processed, each such IPsec header must pass each IPsec check. So an anonymous attacker will not be able to do such an attack.) ICMPv6 After RFC2463 was published, IETF ipngwg has decided to disallow ICMPv6 error packet against ICMPv6 redirect, to prevent ICMPv6 storm on a network medium. This is already implemented into the kernel. Applications For userland programming, we support IPv6 socket API as specified in RFC2553, RFC2292 and upcoming Internet drafts. TCP/UDP over IPv6 is available and quite stable. You can enjoy &man.telnet.1;, &man.ftp.1;, &man.rlogin.1;, &man.rsh.1;, &man.ssh.1;, etc. These applications are protocol independent. That is, they automatically chooses IPv4 or IPv6 according to DNS. Kernel Internals While ip_forward() calls ip_output(), ip6_forward() directly calls if_output() since routers must not divide IPv6 packets into fragments. ICMPv6 should contain the original packet as long as possible up to 1280. UDP6/IP6 port unreach, for instance, should contain all extension headers and the *unchanged* UDP6 and IP6 headers. So, all IP6 functions except TCP never convert network byte order into host byte order, to save the original packet. tcp_input(), udp6_input() and icmp6_input() can not assume that IP6 header is preceding the transport headers due to extension headers. So, in6_cksum() was implemented to handle packets whose IP6 header and transport header is not continuous. TCP/IP6 nor UDP6/IP6 header structures do not exist for checksum calculation. To process IP6 header, extension headers and transport headers easily, network drivers are now required to store packets in one internal mbuf or one or more external mbufs. A typical old driver prepares two internal mbufs for 96 - 204 bytes data, however, now such packet data is stored in one external mbuf. netstat -s -p ip6 tells you whether or not your driver conforms such requirement. In the following example, "cce0" violates the requirement. (For more information, refer to Section 2.) Mbuf statistics: 317 one mbuf two or more mbuf:: lo0 = 8 cce0 = 10 3282 one ext mbuf - 0 two or more ext mbuf - + 0 two or more ext mbuf Each input function calls IP6_EXTHDR_CHECK in the beginning to check if the region between IP6 and its header is continuous. IP6_EXTHDR_CHECK calls m_pullup() only if the mbuf has M_LOOP flag, that is, the packet comes from the loopback interface. m_pullup() is never called for packets coming from physical network interfaces. Both IP and IP6 reassemble functions never call m_pullup(). IPv4 mapped address and IPv6 wildcard socket RFC2553 describes IPv4 mapped address (3.7) and special behavior of IPv6 wildcard bind socket (3.8). The spec allows you to: Accept IPv4 connections by AF_INET6 wildcard bind socket. Transmit IPv4 packet over AF_INET6 socket by using special form of the address like ::ffff:10.1.1.1. but the spec itself is very complicated and does not specify how the socket layer should behave. Here we call the former one "listening side" and the latter one "initiating side", for reference purposes. You can perform wildcard bind on both of the address families, on the same port. The following table show the behavior of FreeBSD 4.x. listening side initiating side (AF_INET6 wildcard (connection to ::ffff:10.1.1.1) socket gets IPv4 conn.) --- --- FreeBSD 4.x configurable supported - default: enabled - + default: enabled The following sections will give you more details, and how you can configure the behavior. Comments on listening side: It looks that RFC2553 talks too little on wildcard bind issue, especially on the port space issue, failure mode and relationship between AF_INET/INET6 wildcard bind. There can be several separate interpretation for this RFC which conform to it but behaves differently. So, to implement portable application you should assume nothing about the behavior in the kernel. Using &man.getaddrinfo.3; is the safest way. Port number space and wildcard bind issues were discussed in detail on ipv6imp mailing list, in mid March 1999 and it looks that there is no concrete consensus (means, up to implementers). You may want to check the mailing list archives. If a server application would like to accept IPv4 and IPv6 connections, there will be two alternatives. One is using AF_INET and AF_INET6 socket (you will need two sockets). Use &man.getaddrinfo.3; with AI_PASSIVE into ai_flags, and &man.socket.2; and &man.bind.2; to all the addresses returned. By opening multiple sockets, you can accept connections onto the socket with proper address family. IPv4 connections will be accepted by AF_INET socket, and IPv6 connections will be accepted by AF_INET6 socket. Another way is using one AF_INET6 wildcard bind socket. Use &man.getaddrinfo.3; with AI_PASSIVE into ai_flags and with AF_INET6 into ai_family, and set the 1st argument hostname to NULL. And &man.socket.2; and &man.bind.2; to the address returned. (should be IPv6 unspecified addr). You can accept either of IPv4 and IPv6 packet via this one socket. To support only IPv6 traffic on AF_INET6 wildcard binded socket portably, always check the peer address when a connection is made toward AF_INET6 listening socket. If the address is IPv4 mapped address, you may want to reject the connection. You can check the condition by using IN6_IS_ADDR_V4MAPPED() macro. To resolve this issue more easily, there is system dependent &man.setsockopt.2; option, IPV6_BINDV6ONLY, used like below. - int on; + int on; setsockopt(s, IPPROTO_IPV6, IPV6_BINDV6ONLY, - (char *)&on, sizeof (on)) < 0)); - + (char *)&on, sizeof (on)) < 0)); When this call succeed, then this socket only receive IPv6 packets. Comments on initiating side: Advise to application implementers: to implement a portable IPv6 application (which works on multiple IPv6 kernels), we believe that the following is the key to the success: NEVER hardcode AF_INET nor AF_INET6. Use &man.getaddrinfo.3; and &man.getnameinfo.3; throughout the system. Never use gethostby*(), getaddrby*(), inet_*() or getipnodeby*(). (To update existing applications to be IPv6 aware easily, sometime getipnodeby*() will be useful. But if possible, try to rewrite the code to use &man.getaddrinfo.3; and &man.getnameinfo.3;.) If you would like to connect to destination, use &man.getaddrinfo.3; and try all the destination returned, like &man.telnet.1; does. Some of the IPv6 stack is shipped with buggy &man.getaddrinfo.3;. Ship a minimal working version with your application and use that as last resort. If you would like to use AF_INET6 socket for both IPv4 and IPv6 outgoing connection, you will need to use &man.getipnodebyname.3;. When you would like to update your existing application to be IPv6 aware with minimal effort, this approach might be chosen. But please note that it is a temporal solution, because &man.getipnodebyname.3; itself is not recommended as it does not handle scoped IPv6 addresses at all. For IPv6 name resolution, &man.getaddrinfo.3; is the preferred API. So you should rewrite your application to use &man.getaddrinfo.3;, when you get the time to do it. When writing applications that make outgoing connections, story goes much simpler if you treat AF_INET and AF_INET6 as totally separate address family. {set,get}sockopt issue goes simpler, DNS issue will be made simpler. We do not recommend you to rely upon IPv4 mapped address. unified tcp and inpcb code FreeBSD 4.x uses shared tcp code between IPv4 and IPv6 (from sys/netinet/tcp*) and separate udp4/6 code. It uses unified inpcb structure. The platform can be configured to support IPv4 mapped address. Kernel configuration is summarized as follows: By default, AF_INET6 socket will grab IPv4 connections in certain condition, and can initiate connection to IPv4 destination embedded in IPv4 mapped IPv6 address. You can disable it on entire system with sysctl like below. sysctl net.inet6.ip6.mapped_addr=0 listening side Each socket can be configured to support special AF_INET6 wildcard bind (enabled by default). You can disable it on each socket basis with &man.setsockopt.2; like below. - int on; + int on; setsockopt(s, IPPROTO_IPV6, IPV6_BINDV6ONLY, - (char *)&on, sizeof (on)) < 0)); - + (char *)&on, sizeof (on)) < 0)); Wildcard AF_INET6 socket grabs IPv4 connection if and only if the following conditions are satisfied: there is no AF_INET socket that matches the IPv4 connection the AF_INET6 socket is configured to accept IPv4 traffic, i.e. getsockopt(IPV6_BINDV6ONLY) returns 0. There is no problem with open/close ordering. initiating side FreeBSD 4.x supports outgoing connection to IPv4 mapped address (::ffff:10.1.1.1), if the node is configured to support IPv4 mapped address. sockaddr_storage When RFC2553 was about to be finalized, there was discussion on how struct sockaddr_storage members are named. One proposal is to prepend "__" to the members (like "__ss_len") as they should not be touched. The other proposal was not to prepend it (like "ss_len") as we need to touch those members directly. There was no clear consensus on it. As a result, RFC2553 defines struct sockaddr_storage as follows: - struct sockaddr_storage { + struct sockaddr_storage { u_char __ss_len; /* address length */ u_char __ss_family; /* address family */ /* and bunch of padding */ - }; - + }; On the contrary, XNET draft defines as follows: - struct sockaddr_storage { + struct sockaddr_storage { u_char ss_len; /* address length */ u_char ss_family; /* address family */ /* and bunch of padding */ - }; - + }; In December 1999, it was agreed that RFC2553bis should pick the latter (XNET) definition. Current implementation conforms to XNET definition, based on RFC2553bis discussion. If you look at multiple IPv6 implementations, you will be able to see both definitions. As an userland programmer, the most portable way of dealing with it is to: ensure ss_family and/or ss_len are available on the platform, by using GNU autoconf, have -Dss_family=__ss_family to unify all occurrences (including header file) into __ss_family, or never touch __ss_family. cast to sockaddr * and use sa_family like: - struct sockaddr_storage ss; - family = ((struct sockaddr *)&ss)->sa_family - + struct sockaddr_storage ss; + family = ((struct sockaddr *)&ss)->sa_family Network Drivers Now following two items are required to be supported by standard drivers: mbuf clustering requirement. In this stable release, we changed MINCLSIZE into MHLEN+1 for all the operating systems in order to make all the drivers behave as we expect. multicast. If &man.ifmcstat.8; yields no multicast group for a interface, that interface has to be patched. If any of the drivers do not support the requirements, then the drivers can not be used for IPv6 and/or IPsec communication. If you find any problem with your card using IPv6/IPsec, then, please report it to the &a.bugs;. (NOTE: In the past we required all PCMCIA drivers to have a call to in6_ifattach(). We have no such requirement any more) Translator We categorize IPv4/IPv6 translator into 4 types: Translator A --- It is used in the early stage of transition to make it possible to establish a connection from an IPv6 host in an IPv6 island to an IPv4 host in the IPv4 ocean. Translator B --- It is used in the early stage of transition to make it possible to establish a connection from an IPv4 host in the IPv4 ocean to an IPv6 host in an IPv6 island. Translator C --- It is used in the late stage of transition to make it possible to establish a connection from an IPv4 host in an IPv4 island to an IPv6 host in the IPv6 ocean. Translator D --- It is used in the late stage of transition to make it possible to establish a connection from an IPv6 host in the IPv6 ocean to an IPv4 host in an IPv4 island. TCP relay translator for category A is supported. This is called "FAITH". We also provide IP header translator for category A. (The latter is not yet put into FreeBSD 4.x yet.) FAITH TCP relay translator FAITH system uses TCP relay daemon called &man.faithd.8; helped by the kernel. FAITH will reserve an IPv6 address prefix, and relay TCP connection toward that prefix to IPv4 destination. For example, if the reserved IPv6 prefix is 2001:0DB8:0200:ffff::, and the IPv6 destination for TCP connection is 2001:0DB8:0200:ffff::163.221.202.12, the connection will be relayed toward IPv4 destination 163.221.202.12. destination IPv4 node (163.221.202.12) ^ | IPv4 tcp toward 163.221.202.12 FAITH-relay dual stack node ^ | IPv6 TCP toward 2001:0DB8:0200:ffff::163.221.202.12 - source IPv6 node - + source IPv6 node &man.faithd.8; must be invoked on FAITH-relay dual stack node. For more details, consult src/usr.sbin/faithd/README IPsec IPsec is mainly organized by three components. Policy Management Key Management AH and ESP handling Policy Management The kernel implements experimental policy management code. There are two way to manage security policy. One is to configure per-socket policy using &man.setsockopt.2;. In this cases, policy configuration is described in &man.ipsec.set.policy.3;. The other is to configure kernel packet filter-based policy using PF_KEY interface, via &man.setkey.8;. The policy entry is not re-ordered with its indexes, so the order of entry when you add is very significant. Key Management The key management code implemented in this kit (sys/netkey) is a home-brew PFKEY v2 implementation. This conforms to RFC2367. The home-brew IKE daemon, "racoon" is included in the kit (kame/kame/racoon). Basically you will need to run racoon as daemon, then set up a policy to require keys (like ping -P 'out ipsec esp/transport//use'). The kernel will contact racoon daemon as necessary to exchange keys. AH and ESP handling IPsec module is implemented as "hooks" to the standard IPv4/IPv6 processing. When sending a packet, ip{,6}_output() checks if ESP/AH processing is required by checking if a matching SPD (Security Policy Database) is found. If ESP/AH is needed, {esp,ah}{4,6}_output() will be called and mbuf will be updated accordingly. When a packet is received, {esp,ah}4_input() will be called based on protocol number, i.e. (*inetsw[proto])(). {esp,ah}4_input() will decrypt/check authenticity of the packet, and strips off daisy-chained header and padding for ESP/AH. It is safe to strip off the ESP/AH header on packet reception, since we will never use the received packet in "as is" form. By using ESP/AH, TCP4/6 effective data segment size will be affected by extra daisy-chained headers inserted by ESP/AH. Our code takes care of the case. Basic crypto functions can be found in directory "sys/crypto". ESP/AH transform are listed in {esp,ah}_core.c with wrapper functions. If you wish to add some algorithm, add wrapper function in {esp,ah}_core.c, and add your crypto algorithm code into sys/crypto. Tunnel mode is partially supported in this release, with the following restrictions: IPsec tunnel is not combined with GIF generic tunneling interface. It needs a great care because we may create an infinite loop between ip_output() and tunnelifp->if_output(). Opinion varies if it is better to unify them, or not. MTU and Don't Fragment bit (IPv4) considerations need more checking, but basically works fine. Authentication model for AH tunnel must be revisited. We will need to improve the policy management engine, eventually. Conformance to RFCs and IDs The IPsec code in the kernel conforms (or, tries to conform) to the following standards: "old IPsec" specification documented in rfc182[5-9].txt "new IPsec" specification documented in rfc240[1-6].txt, rfc241[01].txt, rfc2451.txt and draft-mcdonald-simple-ipsec-api-01.txt (draft expired, but you can take from ftp://ftp.kame.net/pub/internet-drafts/). (NOTE: IKE specifications, rfc241[7-9].txt are implemented in userland, as "racoon" IKE daemon) Currently supported algorithms are: old IPsec AH null crypto checksum (no document, just for debugging) keyed MD5 with 128bit crypto checksum (rfc1828.txt) keyed SHA1 with 128bit crypto checksum (no document) HMAC MD5 with 128bit crypto checksum (rfc2085.txt) HMAC SHA1 with 128bit crypto checksum (no document) old IPsec ESP null encryption (no document, similar to rfc2410.txt) DES-CBC mode (rfc1829.txt) new IPsec AH null crypto checksum (no document, just for debugging) keyed MD5 with 96bit crypto checksum (no document) keyed SHA1 with 96bit crypto checksum (no document) HMAC MD5 with 96bit crypto checksum (rfc2403.txt) HMAC SHA1 with 96bit crypto checksum (rfc2404.txt) new IPsec ESP null encryption (rfc2410.txt) DES-CBC with derived IV (draft-ietf-ipsec-ciph-des-derived-01.txt, draft expired) DES-CBC with explicit IV (rfc2405.txt) 3DES-CBC with explicit IV (rfc2451.txt) BLOWFISH CBC (rfc2451.txt) CAST128 CBC (rfc2451.txt) RC5 CBC (rfc2451.txt) each of the above can be combined with: ESP authentication with HMAC-MD5(96bit) ESP authentication with HMAC-SHA1(96bit) The following algorithms are NOT supported: old IPsec AH HMAC MD5 with 128bit crypto checksum + 64bit replay prevention (rfc2085.txt) keyed SHA1 with 160bit crypto checksum + 32bit padding (rfc1852.txt) IPsec (in kernel) and IKE (in userland as "racoon") has been tested at several interoperability test events, and it is known to interoperate with many other implementations well. Also, current IPsec implementation as quite wide coverage for IPsec crypto algorithms documented in RFC (we cover algorithms without intellectual property issues only). ECN consideration on IPsec tunnels ECN-friendly IPsec tunnel is supported as described in draft-ipsec-ecn-00.txt. Normal IPsec tunnel is described in RFC2401. On encapsulation, IPv4 TOS field (or, IPv6 traffic class field) will be copied from inner IP header to outer IP header. On decapsulation outer IP header will be simply dropped. The decapsulation rule is not compatible with ECN, since ECN bit on the outer IP TOS/traffic class field will be lost. To make IPsec tunnel ECN-friendly, we should modify encapsulation and decapsulation procedure. This is described in http://www.aciri.org/floyd/papers/draft-ipsec-ecn-00.txt, chapter 3. IPsec tunnel implementation can give you three behaviors, by setting net.inet.ipsec.ecn (or net.inet6.ipsec6.ecn) to some value: RFC2401: no consideration for ECN (sysctl value -1) ECN forbidden (sysctl value 0) ECN allowed (sysctl value 1) Note that the behavior is configurable in per-node manner, not per-SA manner (draft-ipsec-ecn-00 wants per-SA configuration, but it looks too much for me). The behavior is summarized as follows (see source code for more detail): encapsulate decapsulate --- --- RFC2401 copy all TOS bits drop TOS bits on outer from inner to outer. (use inner TOS bits as is) ECN forbidden copy TOS bits except for ECN drop TOS bits on outer (masked with 0xfc) from inner (use inner TOS bits as is) to outer. set ECN bits to 0. ECN allowed copy TOS bits except for ECN use inner TOS bits with some CE (masked with 0xfe) from change. if outer ECN CE bit inner to outer. is 1, enable ECN CE bit on - set ECN CE bit to 0. the inner. - - + set ECN CE bit to 0. the inner. General strategy for configuration is as follows: if both IPsec tunnel endpoint are capable of ECN-friendly behavior, you should better configure both end to ECN allowed (sysctl value 1). if the other end is very strict about TOS bit, use "RFC2401" (sysctl value -1). in other cases, use "ECN forbidden" (sysctl value 0). The default behavior is "ECN forbidden" (sysctl value 0). For more information, please refer to: http://www.aciri.org/floyd/papers/draft-ipsec-ecn-00.txt, RFC2481 (Explicit Congestion Notification), src/sys/netinet6/{ah,esp}_input.c (Thanks goes to Kenjiro Cho kjc@csl.sony.co.jp for detailed analysis) Interoperability Here are (some of) platforms that KAME code have tested IPsec/IKE interoperability in the past. Note that both ends may have modified their implementation, so use the following list just for reference purposes. Altiga, Ashley-laurent (vpcom.com), Data Fellows (F-Secure), Ericsson ACC, FreeS/WAN, HITACHI, IBM &aix;, IIJ, Intel, µsoft; &windowsnt;, NIST (linux IPsec + plutoplus), Netscreen, OpenBSD, RedCreek, Routerware, SSH, Secure Computing, Soliton, Toshiba, VPNet, Yamaha RT100i Index: head/en_US.ISO8859-1/books/developers-handbook/tools/chapter.xml =================================================================== --- head/en_US.ISO8859-1/books/developers-handbook/tools/chapter.xml (revision 50804) +++ head/en_US.ISO8859-1/books/developers-handbook/tools/chapter.xml (revision 50805) @@ -1,2269 +1,2254 @@ Programming Tools JamesRaynardContributed by MurrayStokely Synopsis This chapter is an introduction to using some of the programming tools supplied with FreeBSD, although much of it will be applicable to many other versions of &unix;. It does not attempt to describe coding in any detail. Most of the chapter assumes little or no previous programming knowledge, although it is hoped that most programmers will find something of value in it. Introduction FreeBSD offers an excellent development environment. Compilers for C and C++ and an assembler come with the basic system, not to mention classic &unix; tools such as sed and awk. If that is not enough, there are many more compilers and interpreters in the Ports collection. The following section, Introduction to Programming, lists some of the available options. FreeBSD is very compatible with standards such as &posix; and ANSI C, as well with its own BSD heritage, so it is possible to write applications that will compile and run with little or no modification on a wide range of platforms. However, all this power can be rather overwhelming at first if you have never written programs on a &unix; platform before. This document aims to help you get up and running, without getting too deeply into more advanced topics. The intention is that this document should give you enough of the basics to be able to make some sense of the documentation. Most of the document requires little or no knowledge of programming, although it does assume a basic competence with using &unix; and a willingness to learn! Introduction to Programming A program is a set of instructions that tell the computer to do various things; sometimes the instruction it has to perform depends on what happened when it performed a previous instruction. This section gives an overview of the two main ways in which you can give these instructions, or commands as they are usually called. One way uses an interpreter, the other a compiler. As human languages are too difficult for a computer to understand in an unambiguous way, commands are usually written in one or other languages specially designed for the purpose. Interpreters With an interpreter, the language comes as an environment, where you type in commands at a prompt and the environment executes them for you. For more complicated programs, you can type the commands into a file and get the interpreter to load the file and execute the commands in it. If anything goes wrong, many interpreters will drop you into a debugger to help you track down the problem. The advantage of this is that you can see the results of your commands immediately, and mistakes can be corrected readily. The biggest disadvantage comes when you want to share your programs with someone. They must have the same interpreter, or you must have some way of giving it to them, and they need to understand how to use it. Also users may not appreciate being thrown into a debugger if they press the wrong key! From a performance point of view, interpreters can use up a lot of memory, and generally do not generate code as efficiently as compilers. In my opinion, interpreted languages are the best way to start if you have not done any programming before. This kind of environment is typically found with languages like Lisp, Smalltalk, Perl and Basic. It could also be argued that the &unix; shell (sh, csh) is itself an interpreter, and many people do in fact write shell scripts to help with various housekeeping tasks on their machine. Indeed, part of the original &unix; philosophy was to provide lots of small utility programs that could be linked together in shell scripts to perform useful tasks. Interpreters available with FreeBSD Here is a list of interpreters that are available from the &os; Ports Collection, with a brief discussion of some of the more popular interpreted languages. Instructions on how to get and install applications from the Ports Collection can be found in the Ports section of the handbook. BASIC Short for Beginner's All-purpose Symbolic Instruction Code. Developed in the 1950s for teaching University students to program and provided with every self-respecting personal computer in the 1980s, BASIC has been the first programming language for many programmers. It is also the foundation for Visual Basic. The Bywater Basic Interpreter can be found in the Ports Collection as lang/bwbasic and the Phil Cockroft's Basic Interpreter (formerly Rabbit Basic) is available as lang/pbasic. Lisp A language that was developed in the late 1950s as an alternative to the number-crunching languages that were popular at the time. Instead of being based on numbers, Lisp is based on lists; in fact, the name is short for List Processing. It is very popular in AI (Artificial Intelligence) circles. Lisp is an extremely powerful and sophisticated language, but can be rather large and unwieldy. Various implementations of Lisp that can run on &unix; systems are available in the Ports Collection for &os;. GNU Common Lisp can be found as lang/gcl. CLISP by Bruno Haible and Michael Stoll is available as lang/clisp. For CMUCL, which includes a highly-optimizing compiler too, or simpler Lisp implementations like SLisp, which implements most of the Common Lisp constructs in a few hundred lines of C code, lang/cmucl and lang/slisp are available respectively. Perl Very popular with system administrators for writing scripts; also often used on World Wide Web servers for writing CGI scripts. Perl is available in the Ports Collection as lang/perl5.16 for all &os; releases. Scheme A dialect of Lisp that is rather more compact and cleaner than Common Lisp. Popular in Universities as it is simple enough to teach to undergraduates as a first language, while it has a high enough level of abstraction to be used in research work. Scheme is available from the Ports Collection as lang/elk for the Elk Scheme Interpreter. The MIT Scheme Interpreter can be found in lang/mit-scheme and the SCM Scheme Interpreter in lang/scm. Icon Icon is a high-level language with extensive facilities for processing strings and structures. The version of Icon for &os; can be found in the Ports Collection as lang/icon. Logo Logo is a language that is easy to learn, and has been used as an introductory programming language in various courses. It is an excellent tool to work with when teaching programming to smaller age groups, as it makes creation of elaborate geometric shapes an easy task. The latest version of Logo for &os; is available from the Ports Collection in lang/logo. Python Python is an Object-Oriented, interpreted language. Its advocates argue that it is one of the best languages to start programming with, since it is relatively easy to start with, but is not limited in comparison to other popular interpreted languages that are used for the development of large, complex applications (Perl and Tcl are two other languages that are popular for such tasks). The latest version of Python is available from the Ports Collection in lang/python. Ruby Ruby is an interpreter, pure object-oriented programming language. It has become widely popular because of its easy to understand syntax, flexibility when writing code, and the ability to easily develop and maintain large, complex programs. Ruby is available from the Ports Collection as lang/ruby18. Tcl and Tk Tcl is an embeddable, interpreted language, that has become widely used and became popular mostly because of its portability to many platforms. It can be used both for quickly writing small, prototype applications, or (when combined with Tk, a GUI toolkit) fully-fledged, featureful programs. Various versions of Tcl are available as ports for &os;. The latest version, Tcl 8.5, can be found in lang/tcl85. Compilers Compilers are rather different. First of all, you write your code in a file (or files) using an editor. You then run the compiler and see if it accepts your program. If it did not compile, grit your teeth and go back to the editor; if it did compile and gave you a program, you can run it either at a shell command prompt or in a debugger to see if it works properly. If you run it in the shell, you may get a core dump. Obviously, this is not quite as direct as using an interpreter. However it allows you to do a lot of things which are very difficult or even impossible with an interpreter, such as writing code which interacts closely with the operating system—or even writing your own operating system! It is also useful if you need to write very efficient code, as the compiler can take its time and optimize the code, which would not be acceptable in an interpreter. Moreover, distributing a program written for a compiler is usually more straightforward than one written for an interpreter—you can just give them a copy of the executable, assuming they have the same operating system as you. As the edit-compile-run-debug cycle is rather tedious when using separate programs, many commercial compiler makers have produced Integrated Development Environments (IDEs for short). FreeBSD does not include an IDE in the base system, but devel/kdevelop is available in the Ports Collection and many use Emacs for this purpose. Using Emacs as an IDE is discussed in . Compiling with <command>cc</command> This section deals with the gcc and clang compilers for C and C++, since they come with the &os; base system. Starting with &os; 10.X clang is installed as cc. The details of producing a program with an interpreter vary considerably between interpreters, and are usually well covered in the documentation and on-line help for the interpreter. Once you have written your masterpiece, the next step is to convert it into something that will (hopefully!) run on FreeBSD. This usually involves several steps, each of which is done by a separate program. Pre-process your source code to remove comments and do other tricks like expanding macros in C. Check the syntax of your code to see if you have obeyed the rules of the language. If you have not, it will complain! Convert the source code into assembly language—this is very close to machine code, but still understandable by humans. Allegedly. Convert the assembly language into machine code—yep, we are talking bits and bytes, ones and zeros here. Check that you have used things like functions and global variables in a consistent way. For example, if you have called a non-existent function, it will complain. If you are trying to produce an executable from several source code files, work out how to fit them all together. Work out how to produce something that the system's run-time loader will be able to load into memory and run. Finally, write the executable on the filesystem. The word compiling is often used to refer to just steps 1 to 4—the others are referred to as linking. Sometimes step 1 is referred to as pre-processing and steps 3-4 as assembling. Fortunately, almost all this detail is hidden from you, as cc is a front end that manages calling all these programs with the right arguments for you; simply typing &prompt.user; cc foobar.c will cause foobar.c to be compiled by all the steps above. If you have more than one file to compile, just do something like &prompt.user; cc foo.c bar.c Note that the syntax checking is just that—checking the syntax. It will not check for any logical mistakes you may have made, like putting the program into an infinite loop, or using a bubble sort when you meant to use a binary sort. In case you did not know, a binary sort is an efficient way of sorting things into order and a bubble sort is not. There are lots and lots of options for cc, which are all in the manual page. Here are a few of the most important ones, with examples of how to use them. The output name of the file. If you do not use this option, cc will produce an executable called a.out. The reasons for this are buried in the mists of history. &prompt.user; cc foobar.c executable is a.out -&prompt.user; cc -o foobar foobar.c executable is foobar - +&prompt.user; cc -o foobar foobar.c executable is foobar Just compile the file, do not link it. Useful for toy programs where you just want to check the syntax, or if you are using a Makefile. - &prompt.user; cc -c foobar.c - + &prompt.user; cc -c foobar.c This will produce an object file (not an executable) called foobar.o. This can be linked together with other object files into an executable. Create a debug version of the executable. This makes the compiler put information into the executable about which line of which source file corresponds to which function call. A debugger can use this information to show the source code as you step through the program, which is very useful; the disadvantage is that all this extra information makes the program much bigger. Normally, you compile with while you are developing a program and then compile a release version without when you are satisfied it works properly. - &prompt.user; cc -g foobar.c - + &prompt.user; cc -g foobar.c This will produce a debug version of the program. Note, we did not use the flag to specify the executable name, so we will get an executable called a.out. Producing a debug version called foobar is left as an exercise for the reader! Create an optimized version of the executable. The compiler performs various clever tricks to try to produce an executable that runs faster than normal. You can add a number after the to specify a higher level of optimization, but this often exposes bugs in the compiler's optimizer. - &prompt.user; cc -O -o foobar foobar.c - + &prompt.user; cc -O -o foobar foobar.c This will produce an optimized version of foobar. The following three flags will force cc to check that your code complies to the relevant international standard, often referred to as the ANSI standard, though strictly speaking it is an ISO standard. Enable all the warnings which the authors of cc believe are worthwhile. Despite the name, it will not enable all the warnings cc is capable of. Turn off most, but not all, of the non-ANSI C features provided by cc. Despite the name, it does not guarantee strictly that your code will comply to the standard. Turn off all cc's non-ANSI C features. Without these flags, cc will allow you to use some of its non-standard extensions to the standard. Some of these are very useful, but will not work with other compilers—in fact, one of the main aims of the standard is to allow people to write code that will work with any compiler on any system. This is known as portable code. Generally, you should try to make your code as portable as possible, as otherwise you may have to completely rewrite the program later to get it to work somewhere else—and who knows what you may be using in a few years time? &prompt.user; cc -Wall -ansi -pedantic -o foobar foobar.c This will produce an executable foobar after checking foobar.c for standard compliance. Specify a function library to be used at link time. The most common example of this is when compiling a program that uses some of the mathematical functions in C. Unlike most other platforms, these are in a separate library from the standard C one and you have to tell the compiler to add it. The rule is that if the library is called libsomething.a, you give cc the argument . For example, the math library is libm.a, so you give cc the argument . A common gotcha with the math library is that it has to be the last library on the command line. - &prompt.user; cc -o foobar foobar.c -lm - + &prompt.user; cc -o foobar foobar.c -lm This will link the math library functions into foobar. If you are compiling C++ code, use c++. c++ can also be invoked as clang++ on &os;. &prompt.user; c++ -o foobar foobar.cc This will both produce an executable foobar from the C++ source file foobar.cc. Common <command>cc</command> Queries and Problems I am trying to write a program which uses the sin() function and I get an error like this. What does it mean? - /var/tmp/cc0143941.o: Undefined symbol `_sin' referenced from text segment - + /var/tmp/cc0143941.o: Undefined symbol `_sin' referenced from text segment When using mathematical functions like sin(), you have to tell cc to link in the math library, like so: - &prompt.user; cc -o foobar foobar.c -lm - + &prompt.user; cc -o foobar foobar.c -lm All right, I wrote this simple program to practice using . All it does is raise 2.1 to the power of 6. #include <stdio.h> int main() { float f; f = pow(2.1, 6); printf("2.1 ^ 6 = %f\n", f); return 0; } and I compiled it as: - &prompt.user; cc temp.c -lm - + &prompt.user; cc temp.c -lm like you said I should, but I get this when I run it: &prompt.user; ./a.out -2.1 ^ 6 = 1023.000000 - +2.1 ^ 6 = 1023.000000 This is not the right answer! What is going on? When the compiler sees you call a function, it checks if it has already seen a prototype for it. If it has not, it assumes the function returns an int, which is definitely not what you want here. So how do I fix this? The prototypes for the mathematical functions are in math.h. If you include this file, the compiler will be able to find the prototype and it will stop doing strange things to your calculation! #include <math.h> #include <stdio.h> int main() { ... After recompiling it as you did before, run it: &prompt.user; ./a.out -2.1 ^ 6 = 85.766121 - +2.1 ^ 6 = 85.766121 If you are using any of the mathematical functions, always include math.h and remember to link in the math library. I compiled a file called foobar.c and I cannot find an executable called foobar. Where has it gone? Remember, cc will call the executable a.out unless you tell it differently. Use the option: - &prompt.user; cc -o foobar foobar.c - + &prompt.user; cc -o foobar foobar.c OK, I have an executable called foobar, I can see it when I run ls, but when I type in foobar at the command prompt it tells me there is no such file. Why can it not find it? Unlike &ms-dos;, &unix; does not look in the current directory when it is trying to find out which executable you want it to run, unless you tell it to. Type ./foobar, which means run the file called foobar in the current directory. I called my executable test, but nothing happens when I run it. What is going on? Most &unix; systems have a program called test in /usr/bin and the shell is picking that one up before it gets to checking the current directory. Either type: - &prompt.user; ./test - + &prompt.user; ./test or choose a better name for your program! I compiled my program and it seemed to run all right at first, then there was an error and it said something about core dumped. What does that mean? The name core dump dates back to the very early days of &unix;, when the machines used core memory for storing data. Basically, if the program failed under certain conditions, the system would write the contents of core memory to disk in a file called core, which the programmer could then pore over to find out what went wrong. Fascinating stuff, but what I am supposed to do now? Use gdb to analyze the core (see ). When my program dumped core, it said something about a segmentation fault. What is that? This basically means that your program tried to perform some sort of illegal operation on memory; &unix; is designed to protect the operating system and other programs from rogue programs. Common causes for this are: Trying to write to a NULL pointer, eg char *foo = NULL; strcpy(foo, "bang!"); Using a pointer that has not been initialized, eg char *foo; strcpy(foo, "bang!"); The pointer will have some random value that, with luck, will point into an area of memory that is not available to your program and the kernel will kill your program before it can do any damage. If you are unlucky, it will point somewhere inside your own program and corrupt one of your data structures, causing the program to fail mysteriously. Trying to access past the end of an array, eg int bar[20]; bar[27] = 6; Trying to store something in read-only memory, eg char *foo = "My string"; strcpy(foo, "bang!"); &unix; compilers often put string literals like "My string" into read-only areas of memory. Doing naughty things with malloc() and free(), eg char bar[80]; free(bar); or char *foo = malloc(27); free(foo); free(foo); Making one of these mistakes will not always lead to an error, but they are always bad practice. Some systems and compilers are more tolerant than others, which is why programs that ran well on one system can crash when you try them on an another. Sometimes when I get a core dump it says bus error. It says in my &unix; book that this means a hardware problem, but the computer still seems to be working. Is this true? No, fortunately not (unless of course you really do have a hardware problem…). This is usually another way of saying that you accessed memory in a way you should not have. This dumping core business sounds as though it could be quite useful, if I can make it happen when I want to. Can I do this, or do I have to wait until there is an error? Yes, just go to another console or xterm, do - &prompt.user; ps - + &prompt.user; ps to find out the process ID of your program, and do - &prompt.user; kill -ABRT pid - + &prompt.user; kill -ABRT pid where pid is the process ID you looked up. This is useful if your program has got stuck in an infinite loop, for instance. If your program happens to trap SIGABRT, there are several other signals which have a similar effect. Alternatively, you can create a core dump from inside your program, by calling the abort() function. See the manual page of &man.abort.3; to learn more. If you want to create a core dump from outside your program, but do not want the process to terminate, you can use the gcore program. See the manual page of &man.gcore.1; for more information. Make What is <command>make</command>? When you are working on a simple program with only one or two source files, typing in &prompt.user; cc file1.c file2.c is not too bad, but it quickly becomes very tedious when there are several files—and it can take a while to compile, too. One way to get around this is to use object files and only recompile the source file if the source code has changed. So we could have something like: &prompt.user; cc file1.o file2.o … file37.c … if we had changed file37.c, but not any of the others, since the last time we compiled. This may speed up the compilation quite a bit, but does not solve the typing problem. Or we could write a shell script to solve the typing problem, but it would have to re-compile everything, making it very inefficient on a large project. What happens if we have hundreds of source files lying about? What if we are working in a team with other people who forget to tell us when they have changed one of their source files that we use? Perhaps we could put the two solutions together and write something like a shell script that would contain some kind of magic rule saying when a source file needs compiling. Now all we need now is a program that can understand these rules, as it is a bit too complicated for the shell. This program is called make. It reads in a file, called a makefile, that tells it how different files depend on each other, and works out which files need to be re-compiled and which ones do not. For example, a rule could say something like if fromboz.o is older than fromboz.c, that means someone must have changed fromboz.c, so it needs to be re-compiled. The makefile also has rules telling make how to re-compile the source file, making it a much more powerful tool. Makefiles are typically kept in the same directory as the source they apply to, and can be called makefile, Makefile or MAKEFILE. Most programmers use the name Makefile, as this puts it near the top of a directory listing, where it can easily be seen. They do not use the MAKEFILE form as block capitals are often used for documentation files like README. Example of using <command>make</command> Here is a very simple make file: foo: foo.c cc -o foo foo.c It consists of two lines, a dependency line and a creation line. The dependency line here consists of the name of the program (known as the target), followed by a colon, then whitespace, then the name of the source file. When make reads this line, it looks to see if foo exists; if it exists, it compares the time foo was last modified to the time foo.c was last modified. If foo does not exist, or is older than foo.c, it then looks at the creation line to find out what to do. In other words, this is the rule for working out when foo.c needs to be re-compiled. The creation line starts with a tab (press the tab key) and then the command you would type to create foo if you were doing it at a command prompt. If foo is out of date, or does not exist, make then executes this command to create it. In other words, this is the rule which tells make how to re-compile foo.c. So, when you type make, it will make sure that foo is up to date with respect to your latest changes to foo.c. This principle can be extended to Makefiles with hundreds of targets—in fact, on FreeBSD, it is possible to compile the entire operating system just by typing make world in the appropriate directory! Another useful property of makefiles is that the targets do not have to be programs. For instance, we could have a make file that looks like this: foo: foo.c cc -o foo foo.c install: cp foo /home/me We can tell make which target we want to make by typing: &prompt.user; make target make will then only look at that target and ignore any others. For example, if we type make foo with the makefile above, make will ignore the install target. If we just type make on its own, make will always look at the first target and then stop without looking at any others. So if we typed make here, it will just go to the foo target, re-compile foo if necessary, and then stop without going on to the install target. Notice that the install target does not actually depend on anything! This means that the command on the following line is always executed when we try to make that target by typing make install. In this case, it will copy foo into the user's home directory. This is often used by application makefiles, so that the application can be installed in the correct directory when it has been correctly compiled. This is a slightly confusing subject to try to explain. If you do not quite understand how make works, the best thing to do is to write a simple program like hello world and a make file like the one above and experiment. Then progress to using more than one source file, or having the source file include a header file. The touch command is very useful here—it changes the date on a file without you having to edit it. Make and include-files C code often starts with a list of files to include, for example stdio.h. Some of these files are system-include files, some of them are from the project you are now working on: #include <stdio.h> #include "foo.h" int main(.... To make sure that this file is recompiled the moment foo.h is changed, you have to add it in your Makefile: foo: foo.c foo.h The moment your project is getting bigger and you have more and more own include-files to maintain, it will be a pain to keep track of all include files and the files which are depending on it. If you change an include-file but forget to recompile all the files which are depending on it, the results will be devastating. clang has an option to analyze your files and to produce a list of include-files and their dependencies: . If you add this to your Makefile: depend: cc -E -MM *.c > .depend and run make depend, the file .depend will appear with a list of object-files, C-files and the include-files: foo.o: foo.c foo.h If you change foo.h, next time you run make all files depending on foo.h will be recompiled. Do not forget to run make depend each time you add an include-file to one of your files. FreeBSD Makefiles Makefiles can be rather complicated to write. Fortunately, BSD-based systems like FreeBSD come with some very powerful ones as part of the system. One very good example of this is the FreeBSD ports system. Here is the essential part of a typical ports Makefile: MASTER_SITES= ftp://freefall.cdrom.com/pub/FreeBSD/LOCAL_PORTS/ DISTFILES= scheme-microcode+dist-7.3-freebsd.tgz .include <bsd.port.mk> Now, if we go to the directory for this port and type make, the following happens: A check is made to see if the source code for this port is already on the system. If it is not, an FTP connection to the URL in MASTER_SITES is set up to download the source. The checksum for the source is calculated and compared it with one for a known, good, copy of the source. This is to make sure that the source was not corrupted while in transit. Any changes required to make the source work on FreeBSD are applied—this is known as patching. Any special configuration needed for the source is done. (Many &unix; program distributions try to work out which version of &unix; they are being compiled on and which optional &unix; features are present—this is where they are given the information in the FreeBSD ports scenario). The source code for the program is compiled. In effect, we change to the directory where the source was unpacked and do make—the program's own make file has the necessary information to build the program. We now have a compiled version of the program. If we wish, we can test it now; when we feel confident about the program, we can type make install. This will cause the program and any supporting files it needs to be copied into the correct location; an entry is also made into a package database, so that the port can easily be uninstalled later if we change our mind about it. Now I think you will agree that is rather impressive for a four line script! The secret lies in the last line, which tells make to look in the system makefile called bsd.port.mk. It is easy to overlook this line, but this is where all the clever stuff comes from—someone has written a makefile that tells make to do all the things above (plus a couple of other things I did not mention, including handling any errors that may occur) and anyone can get access to that just by putting a single line in their own make file! If you want to have a look at these system makefiles, they are in /usr/share/mk, but it is probably best to wait until you have had a bit of practice with makefiles, as they are very complicated (and if you do look at them, make sure you have a flask of strong coffee handy!) More advanced uses of <command>make</command> Make is a very powerful tool, and can do much more than the simple example above shows. Unfortunately, there are several different versions of make, and they all differ considerably. The best way to learn what they can do is probably to read the documentation—hopefully this introduction will have given you a base from which you can do this. The version of make that comes with FreeBSD is the Berkeley make; there is a tutorial for it in /usr/share/doc/psd/12.make. To view it, do &prompt.user; zmore paper.ascii.gz in that directory. Many applications in the ports use GNU make, which has a very good set of info pages. If you have installed any of these ports, GNU make will automatically have been installed as gmake. It is also available as a port and package in its own right. To view the info pages for GNU make, you will have to edit the dir file in the /usr/local/info directory to add an entry for it. This involves adding a line like * Make: (make). The GNU Make utility. to the file. Once you have done this, you can type info and then select make from the menu (or in Emacs, do C-h i). Debugging The Debugger The debugger that comes with FreeBSD is called gdb (GNU debugger). You start it up by typing &prompt.user; gdb progname although many people prefer to run it inside Emacs. You can do this by: M-x gdb RET progname RET Using a debugger allows you to run the program under more controlled circumstances. Typically, you can step through the program a line at a time, inspect the value of variables, change them, tell the debugger to run up to a certain point and then stop, and so on. You can even attach to a program that is already running, or load a core file to investigate why the program crashed. It is even possible to debug the kernel, though that is a little trickier than the user applications we will be discussing in this section. gdb has quite good on-line help, as well as a set of info pages, so this section will concentrate on a few of the basic commands. Finally, if you find its text-based command-prompt style off-putting, there is a graphical front-end for it (devel/xxgdb) in the Ports Collection. This section is intended to be an introduction to using gdb and does not cover specialized topics such as debugging the kernel. Running a program in the debugger You will need to have compiled the program with the option to get the most out of using gdb. It will work without, but you will only see the name of the function you are in, instead of the source code. If you see a line like: … (no debugging symbols found) … when gdb starts up, you will know that the program was not compiled with the option. At the gdb prompt, type break main. This will tell the debugger that you are not interested in watching the preliminary set-up code in the program being run, and that it should stop execution at the beginning of your code. Now type run to start the program—it will start at the beginning of the set-up code and then get stopped by the debugger when it calls main(). (If you have ever wondered where main() gets called from, now you know!). You can now step through the program, a line at a time, by pressing n. If you get to a function call, you can step into it by pressing s. Once you are in a function call, you can return from stepping into a function call by pressing f. You can also use up and down to take a quick look at the caller. Here is a simple example of how to spot a mistake in a program with gdb. This is our program (with a deliberate mistake): #include <stdio.h> int bazz(int anint); main() { int i; printf("This is my program\n"); bazz(i); return 0; } int bazz(int anint) { printf("You gave me %d\n", anint); return anint; } This program sets i to be 5 and passes it to a function bazz() which prints out the number we gave it. When we compile and run the program we get &prompt.user; cc -g -o temp temp.c &prompt.user; ./temp This is my program anint = 4231 That was not what we expected! Time to see what is going on! &prompt.user; gdb temp GDB is free software and you are welcome to distribute copies of it under certain conditions; type "show copying" to see the conditions. There is absolutely no warranty for GDB; type "show warranty" for details. GDB 4.13 (i386-unknown-freebsd), Copyright 1994 Free Software Foundation, Inc. (gdb) break main Skip the set-up code Breakpoint 1 at 0x160f: file temp.c, line 9. gdb puts breakpoint at main() (gdb) run Run as far as main() Starting program: /home/james/tmp/temp Program starts running Breakpoint 1, main () at temp.c:9 gdb stops at main() (gdb) n Go to next line This is my program Program prints out (gdb) s step into bazz() bazz (anint=4231) at temp.c:17 gdb displays stack frame (gdb) Hang on a minute! How did anint get to be 4231? Did we not we set it to be 5 in main()? Let's move up to main() and have a look. (gdb) up Move up call stack #1 0x1625 in main () at temp.c:11 gdb displays stack frame (gdb) p i Show us the value of i $1 = 4231 gdb displays 4231 Oh dear! Looking at the code, we forgot to initialize i. We meant to put … main() { int i; i = 5; printf("This is my program\n"); … but we left the i=5; line out. As we did not initialize i, it had whatever number happened to be in that area of memory when the program ran, which in this case happened to be 4231. gdb displays the stack frame every time we go into or out of a function, even if we are using up and down to move around the call stack. This shows the name of the function and the values of its arguments, which helps us keep track of where we are and what is going on. (The stack is a storage area where the program stores information about the arguments passed to functions and where to go when it returns from a function call). Examining a core file A core file is basically a file which contains the complete state of the process when it crashed. In the good old days, programmers had to print out hex listings of core files and sweat over machine code manuals, but now life is a bit easier. Incidentally, under FreeBSD and other 4.4BSD systems, a core file is called progname.core instead of just core, to make it clearer which program a core file belongs to. To examine a core file, start up gdb in the usual way. Instead of typing break or run, type (gdb) core progname.core If you are not in the same directory as the core file, you will have to do dir /path/to/core/file first. You should see something like this: &prompt.user; gdb a.out GDB is free software and you are welcome to distribute copies of it under certain conditions; type "show copying" to see the conditions. There is absolutely no warranty for GDB; type "show warranty" for details. GDB 4.13 (i386-unknown-freebsd), Copyright 1994 Free Software Foundation, Inc. (gdb) core a.out.core Core was generated by `a.out'. Program terminated with signal 11, Segmentation fault. Cannot access memory at address 0x7020796d. #0 0x164a in bazz (anint=0x5) at temp.c:17 (gdb) In this case, the program was called a.out, so the core file is called a.out.core. We can see that the program crashed due to trying to access an area in memory that was not available to it in a function called bazz. Sometimes it is useful to be able to see how a function was called, as the problem could have occurred a long way up the call stack in a complex program. The bt command causes gdb to print out a back-trace of the call stack: (gdb) bt #0 0x164a in bazz (anint=0x5) at temp.c:17 #1 0xefbfd888 in end () #2 0x162c in main () at temp.c:11 (gdb) The end() function is called when a program crashes; in this case, the bazz() function was called from main(). Attaching to a running program One of the neatest features about gdb is that it can attach to a program that is already running. Of course, that assumes you have sufficient permissions to do so. A common problem is when you are stepping through a program that forks, and you want to trace the child, but the debugger will only let you trace the parent. What you do is start up another gdb, use ps to find the process ID for the child, and do (gdb) attach pid in gdb, and then debug as usual. That is all very well, you are probably thinking, but by the time I have done that, the child process will be over the hill and far away. Fear not, gentle reader, here is how to do it (courtesy of the gdb info pages): - … + … if ((pid = fork()) < 0) /* _Always_ check this */ error(); else if (pid == 0) { /* child */ int PauseMode = 1; while (PauseMode) sleep(10); /* Wait until someone attaches to us */ … } else { /* parent */ - … + … Now all you have to do is attach to the child, set PauseMode to 0, and wait for the sleep() call to return! Using Emacs as a Development Environment Emacs Emacs is a highly customizable editor—indeed, it has been customized to the point where it is more like an operating system than an editor! Many developers and sysadmins do in fact spend practically all their time working inside Emacs, leaving it only to log out. It is impossible even to summarize everything Emacs can do here, but here are some of the features of interest to developers: Very powerful editor, allowing search-and-replace on both strings and regular expressions (patterns), jumping to start/end of block expression, etc, etc. Pull-down menus and online help. Language-dependent syntax highlighting and indentation. Completely customizable. You can compile and debug programs within Emacs. On a compilation error, you can jump to the offending line of source code. Friendly-ish front-end to the info program used for reading GNU hypertext documentation, including the documentation on Emacs itself. Friendly front-end to gdb, allowing you to look at the source code as you step through your program. And doubtless many more that have been overlooked. Emacs can be installed on &os; using the editors/emacs port. Once it is installed, start it up and do C-h t to read an Emacs tutorial—that means hold down the control key, press h, let go of the control key, and then press t. (Alternatively, you can use the mouse to select Emacs Tutorial from the Help menu.) Although Emacs does have menus, it is well worth learning the key bindings, as it is much quicker when you are editing something to press a couple of keys than to try to find the mouse and then click on the right place. And, when you are talking to seasoned Emacs users, you will find they often casually throw around expressions like M-x replace-s RET foo RET bar RET so it is useful to know what they mean. And in any case, Emacs has far too many useful functions for them to all fit on the menu bars. Fortunately, it is quite easy to pick up the key-bindings, as they are displayed next to the menu item. My advice is to use the menu item for, say, opening a file until you understand how it works and feel confident with it, then try doing C-x C-f. When you are happy with that, move on to another menu command. If you can not remember what a particular combination of keys does, select Describe Key from the Help menu and type it in—Emacs will tell you what it does. You can also use the Command Apropos menu item to find out all the commands which contain a particular word in them, with the key binding next to it. By the way, the expression above means hold down the Meta key, press x, release the Meta key, type replace-s (short for replace-string—another feature of Emacs is that you can abbreviate commands), press the return key, type foo (the string you want replaced), press the return key, type bar (the string you want to replace foo with) and press return again. Emacs will then do the search-and-replace operation you have just requested. If you are wondering what on earth the Meta key is, it is a special key that many &unix; workstations have. Unfortunately, PC's do not have one, so it is usually the alt key (or if you are unlucky, the escape key). Oh, and to get out of Emacs, do C-x C-c (that means hold down the control key, press x, press c and release the control key). If you have any unsaved files open, Emacs will ask you if you want to save them. (Ignore the bit in the documentation where it says C-z is the usual way to leave Emacs—that leaves Emacs hanging around in the background, and is only really useful if you are on a system which does not have virtual terminals). Configuring Emacs Emacs does many wonderful things; some of them are built in, some of them need to be configured. Instead of using a proprietary macro language for configuration, Emacs uses a version of Lisp specially adapted for editors, known as Emacs Lisp. Working with Emacs Lisp can be quite helpful if you want to go on and learn something like Common Lisp. Emacs Lisp has many features of Common Lisp, although it is considerably smaller (and thus easier to master). The best way to learn Emacs Lisp is to download the Emacs Tutorial However, there is no need to actually know any Lisp to get started with configuring Emacs, as I have included a sample .emacs file, which should be enough to get you started. Just copy it into your home directory and restart Emacs if it is already running; it will read the commands from the file and (hopefully) give you a useful basic setup. A sample <filename>.emacs</filename> file Unfortunately, there is far too much here to explain it in detail; however there are one or two points worth mentioning. Everything beginning with a ; is a comment and is ignored by Emacs. In the first line, the -*- Emacs-Lisp -*- is so that we can edit the .emacs file itself within Emacs and get all the fancy features for editing Emacs Lisp. Emacs usually tries to guess this based on the filename, and may not get it right for .emacs. The tab key is bound to an indentation function in some modes, so when you press the tab key, it will indent the current line of code. If you want to put a tab character in whatever you are writing, hold the control key down while you are pressing the tab key. This file supports syntax highlighting for C, C++, Perl, Lisp and Scheme, by guessing the language from the filename. Emacs already has a pre-defined function called next-error. In a compilation output window, this allows you to move from one compilation error to the next by doing M-n; we define a complementary function, previous-error, that allows you to go to a previous error by doing M-p. The nicest feature of all is that C-c C-c will open up the source file in which the error occurred and jump to the appropriate line. We enable Emacs's ability to act as a server, so that if you are doing something outside Emacs and you want to edit a file, you can just type in - &prompt.user; emacsclient filename - + &prompt.user; emacsclient filename and then you can edit the file in your Emacs! Many Emacs users set their EDITOR environment to emacsclient so this happens every time they need to edit a file. A sample <filename>.emacs</filename> file ;; -*-Emacs-Lisp-*- ;; This file is designed to be re-evaled; use the variable first-time ;; to avoid any problems with this. (defvar first-time t "Flag signifying this is the first time that .emacs has been evaled") ;; Meta (global-set-key "\M- " 'set-mark-command) (global-set-key "\M-\C-h" 'backward-kill-word) (global-set-key "\M-\C-r" 'query-replace) (global-set-key "\M-r" 'replace-string) (global-set-key "\M-g" 'goto-line) (global-set-key "\M-h" 'help-command) ;; Function keys (global-set-key [f1] 'manual-entry) (global-set-key [f2] 'info) (global-set-key [f3] 'repeat-complex-command) (global-set-key [f4] 'advertised-undo) (global-set-key [f5] 'eval-current-buffer) (global-set-key [f6] 'buffer-menu) (global-set-key [f7] 'other-window) (global-set-key [f8] 'find-file) (global-set-key [f9] 'save-buffer) (global-set-key [f10] 'next-error) (global-set-key [f11] 'compile) (global-set-key [f12] 'grep) (global-set-key [C-f1] 'compile) (global-set-key [C-f2] 'grep) (global-set-key [C-f3] 'next-error) (global-set-key [C-f4] 'previous-error) (global-set-key [C-f5] 'display-faces) (global-set-key [C-f8] 'dired) (global-set-key [C-f10] 'kill-compilation) ;; Keypad bindings (global-set-key [up] "\C-p") (global-set-key [down] "\C-n") (global-set-key [left] "\C-b") (global-set-key [right] "\C-f") (global-set-key [home] "\C-a") (global-set-key [end] "\C-e") (global-set-key [prior] "\M-v") (global-set-key [next] "\C-v") (global-set-key [C-up] "\M-\C-b") (global-set-key [C-down] "\M-\C-f") (global-set-key [C-left] "\M-b") (global-set-key [C-right] "\M-f") (global-set-key [C-home] "\M-<") (global-set-key [C-end] "\M->") (global-set-key [C-prior] "\M-<") (global-set-key [C-next] "\M->") ;; Mouse (global-set-key [mouse-3] 'imenu) ;; Misc (global-set-key [C-tab] "\C-q\t") ; Control tab quotes a tab. (setq backup-by-copying-when-mismatch t) ;; Treat 'y' or <CR> as yes, 'n' as no. (fset 'yes-or-no-p 'y-or-n-p) (define-key query-replace-map [return] 'act) (define-key query-replace-map [?\C-m] 'act) ;; Load packages (require 'desktop) (require 'tar-mode) ;; Pretty diff mode (autoload 'ediff-buffers "ediff" "Intelligent Emacs interface to diff" t) (autoload 'ediff-files "ediff" "Intelligent Emacs interface to diff" t) (autoload 'ediff-files-remote "ediff" "Intelligent Emacs interface to diff") (if first-time (setq auto-mode-alist (append '(("\\.cpp$" . c++-mode) ("\\.hpp$" . c++-mode) ("\\.lsp$" . lisp-mode) ("\\.scm$" . scheme-mode) ("\\.pl$" . perl-mode) ) auto-mode-alist))) ;; Auto font lock mode (defvar font-lock-auto-mode-list (list 'c-mode 'c++-mode 'c++-c-mode 'emacs-lisp-mode 'lisp-mode 'perl-mode 'scheme-mode) "List of modes to always start in font-lock-mode") (defvar font-lock-mode-keyword-alist '((c++-c-mode . c-font-lock-keywords) (perl-mode . perl-font-lock-keywords)) "Associations between modes and keywords") (defun font-lock-auto-mode-select () "Automatically select font-lock-mode if the current major mode is in font-lock-auto-mode-list" (if (memq major-mode font-lock-auto-mode-list) (progn (font-lock-mode t)) ) ) (global-set-key [M-f1] 'font-lock-fontify-buffer) ;; New dabbrev stuff ;(require 'new-dabbrev) (setq dabbrev-always-check-other-buffers t) (setq dabbrev-abbrev-char-regexp "\\sw\\|\\s_") (add-hook 'emacs-lisp-mode-hook '(lambda () (set (make-local-variable 'dabbrev-case-fold-search) nil) (set (make-local-variable 'dabbrev-case-replace) nil))) (add-hook 'c-mode-hook '(lambda () (set (make-local-variable 'dabbrev-case-fold-search) nil) (set (make-local-variable 'dabbrev-case-replace) nil))) (add-hook 'text-mode-hook '(lambda () (set (make-local-variable 'dabbrev-case-fold-search) t) (set (make-local-variable 'dabbrev-case-replace) t))) ;; C++ and C mode... (defun my-c++-mode-hook () (setq tab-width 4) (define-key c++-mode-map "\C-m" 'reindent-then-newline-and-indent) (define-key c++-mode-map "\C-ce" 'c-comment-edit) (setq c++-auto-hungry-initial-state 'none) (setq c++-delete-function 'backward-delete-char) (setq c++-tab-always-indent t) (setq c-indent-level 4) (setq c-continued-statement-offset 4) (setq c++-empty-arglist-indent 4)) (defun my-c-mode-hook () (setq tab-width 4) (define-key c-mode-map "\C-m" 'reindent-then-newline-and-indent) (define-key c-mode-map "\C-ce" 'c-comment-edit) (setq c-auto-hungry-initial-state 'none) (setq c-delete-function 'backward-delete-char) (setq c-tab-always-indent t) ;; BSD-ish indentation style (setq c-indent-level 4) (setq c-continued-statement-offset 4) (setq c-brace-offset -4) (setq c-argdecl-indent 0) (setq c-label-offset -4)) ;; Perl mode (defun my-perl-mode-hook () (setq tab-width 4) (define-key c++-mode-map "\C-m" 'reindent-then-newline-and-indent) (setq perl-indent-level 4) (setq perl-continued-statement-offset 4)) ;; Scheme mode... (defun my-scheme-mode-hook () (define-key scheme-mode-map "\C-m" 'reindent-then-newline-and-indent)) ;; Emacs-Lisp mode... (defun my-lisp-mode-hook () (define-key lisp-mode-map "\C-m" 'reindent-then-newline-and-indent) (define-key lisp-mode-map "\C-i" 'lisp-indent-line) (define-key lisp-mode-map "\C-j" 'eval-print-last-sexp)) ;; Add all of the hooks... (add-hook 'c++-mode-hook 'my-c++-mode-hook) (add-hook 'c-mode-hook 'my-c-mode-hook) (add-hook 'scheme-mode-hook 'my-scheme-mode-hook) (add-hook 'emacs-lisp-mode-hook 'my-lisp-mode-hook) (add-hook 'lisp-mode-hook 'my-lisp-mode-hook) (add-hook 'perl-mode-hook 'my-perl-mode-hook) ;; Complement to next-error (defun previous-error (n) "Visit previous compilation error message and corresponding source code." (interactive "p") (next-error (- n))) ;; Misc... (transient-mark-mode 1) (setq mark-even-if-inactive t) (setq visible-bell nil) (setq next-line-add-newlines nil) (setq compile-command "make") (setq suggest-key-bindings nil) (put 'eval-expression 'disabled nil) (put 'narrow-to-region 'disabled nil) (put 'set-goal-column 'disabled nil) (if (>= emacs-major-version 21) (setq show-trailing-whitespace t)) ;; Elisp archive searching (autoload 'format-lisp-code-directory "lispdir" nil t) (autoload 'lisp-dir-apropos "lispdir" nil t) (autoload 'lisp-dir-retrieve "lispdir" nil t) (autoload 'lisp-dir-verify "lispdir" nil t) ;; Font lock mode (defun my-make-face (face color &optional bold) "Create a face from a color and optionally make it bold" (make-face face) (copy-face 'default face) (set-face-foreground face color) (if bold (make-face-bold face)) ) (if (eq window-system 'x) (progn (my-make-face 'blue "blue") (my-make-face 'red "red") (my-make-face 'green "dark green") (setq font-lock-comment-face 'blue) (setq font-lock-string-face 'bold) (setq font-lock-type-face 'bold) (setq font-lock-keyword-face 'bold) (setq font-lock-function-name-face 'red) (setq font-lock-doc-string-face 'green) (add-hook 'find-file-hooks 'font-lock-auto-mode-select) (setq baud-rate 1000000) (global-set-key "\C-cmm" 'menu-bar-mode) (global-set-key "\C-cms" 'scroll-bar-mode) (global-set-key [backspace] 'backward-delete-char) ; (global-set-key [delete] 'delete-char) (standard-display-european t) (load-library "iso-transl"))) ;; X11 or PC using direct screen writes (if window-system (progn ;; (global-set-key [M-f1] 'hilit-repaint-command) ;; (global-set-key [M-f2] [?\C-u M-f1]) (setq hilit-mode-enable-list '(not text-mode c-mode c++-mode emacs-lisp-mode lisp-mode scheme-mode) hilit-auto-highlight nil hilit-auto-rehighlight 'visible hilit-inhibit-hooks nil hilit-inhibit-rebinding t) (require 'hilit19) (require 'paren)) (setq baud-rate 2400) ; For slow serial connections ) ;; TTY type terminal (if (and (not window-system) (not (equal system-type 'ms-dos))) (progn (if first-time (progn (keyboard-translate ?\C-h ?\C-?) (keyboard-translate ?\C-? ?\C-h))))) ;; Under UNIX (if (not (equal system-type 'ms-dos)) (progn (if first-time (server-start)))) ;; Add any face changes here (add-hook 'term-setup-hook 'my-term-setup-hook) (defun my-term-setup-hook () (if (eq window-system 'pc) (progn ;; (set-face-background 'default "red") ))) ;; Restore the "desktop" - do this as late as possible (if first-time (progn (desktop-load-default) (desktop-read))) ;; Indicate that this file has been read at least once (setq first-time nil) ;; No need to debug anything now (setq debug-on-error nil) ;; All done (message "All done, %s%s" (user-login-name) ".") Extending the Range of Languages Emacs Understands Now, this is all very well if you only want to program in the languages already catered for in the .emacs file (C, C++, Perl, Lisp and Scheme), but what happens if a new language called whizbang comes out, full of exciting features? The first thing to do is find out if whizbang comes with any files that tell Emacs about the language. These usually end in .el, short for Emacs Lisp. For example, if whizbang is a FreeBSD port, we can locate these files by doing &prompt.user; find /usr/ports/lang/whizbang -name "*.el" -print and install them by copying them into the Emacs site Lisp directory. On &os;, this is /usr/local/share/emacs/site-lisp. So for example, if the output from the find command was /usr/ports/lang/whizbang/work/misc/whizbang.el we would do &prompt.root; cp /usr/ports/lang/whizbang/work/misc/whizbang.el /usr/local/share/emacs/site-lisp Next, we need to decide what extension whizbang source files have. Let's say for the sake of argument that they all end in .wiz. We need to add an entry to our .emacs file to make sure Emacs will be able to use the information in whizbang.el. Find the auto-mode-alist entry in .emacs and add a line for whizbang, such as: … ("\\.lsp$" . lisp-mode) ("\\.wiz$" . whizbang-mode) ("\\.scm$" . scheme-mode) … This means that Emacs will automatically go into whizbang-mode when you edit a file ending in .wiz. Just below this, you will find the font-lock-auto-mode-list entry. Add whizbang-mode to it like so: ;; Auto font lock mode (defvar font-lock-auto-mode-list (list 'c-mode 'c++-mode 'c++-c-mode 'emacs-lisp-mode 'whizbang-mode 'lisp-mode 'perl-mode 'scheme-mode) "List of modes to always start in font-lock-mode") This means that Emacs will always enable font-lock-mode (ie syntax highlighting) when editing a .wiz file. And that is all that is needed. If there is anything else you want done automatically when you open up a .wiz file, you can add a whizbang-mode hook (see my-scheme-mode-hook for a simple example that adds auto-indent). Further Reading For information about setting up a development environment for contributing fixes to FreeBSD itself, please see &man.development.7;. Brian Harvey and Matthew Wright Simply Scheme MIT 1994. ISBN 0-262-08226-8 Randall Schwartz Learning Perl O'Reilly 1993 ISBN 1-56592-042-2 Patrick Henry Winston and Berthold Klaus Paul Horn Lisp (3rd Edition) Addison-Wesley 1989 ISBN 0-201-08319-1 Brian W. Kernighan and Rob Pike The Unix Programming Environment Prentice-Hall 1984 ISBN 0-13-937681-X Brian W. Kernighan and Dennis M. Ritchie The C Programming Language (2nd Edition) Prentice-Hall 1988 ISBN 0-13-110362-8 Bjarne Stroustrup The C++ Programming Language Addison-Wesley 1991 ISBN 0-201-53992-6 W. Richard Stevens Advanced Programming in the Unix Environment Addison-Wesley 1992 ISBN 0-201-56317-7 W. Richard Stevens Unix Network Programming Prentice-Hall 1990 ISBN 0-13-949876-1 Index: head/en_US.ISO8859-1/books/handbook/network-servers/chapter.xml =================================================================== --- head/en_US.ISO8859-1/books/handbook/network-servers/chapter.xml (revision 50804) +++ head/en_US.ISO8859-1/books/handbook/network-servers/chapter.xml (revision 50805) @@ -1,5788 +1,5786 @@ Network Servers Synopsis This chapter covers some of the more frequently used network services on &unix; systems. This includes installing, configuring, testing, and maintaining many different types of network services. Example configuration files are included throughout this chapter for reference. By the end of this chapter, readers will know: How to manage the inetd daemon. How to set up the Network File System (NFS). How to set up the Network Information Server (NIS) for centralizing and sharing user accounts. How to set &os; up to act as an LDAP server or client How to set up automatic network settings using DHCP. How to set up a Domain Name Server (DNS). How to set up the Apache HTTP Server. How to set up a File Transfer Protocol (FTP) server. How to set up a file and print server for &windows; clients using Samba. How to synchronize the time and date, and set up a time server using the Network Time Protocol (NTP). How to set up iSCSI. This chapter assumes a basic knowledge of: /etc/rc scripts. Network terminology. Installation of additional third-party software (). The <application>inetd</application> Super-Server The &man.inetd.8; daemon is sometimes referred to as a Super-Server because it manages connections for many services. Instead of starting multiple applications, only the inetd service needs to be started. When a connection is received for a service that is managed by inetd, it determines which program the connection is destined for, spawns a process for that program, and delegates the program a socket. Using inetd for services that are not heavily used can reduce system load, when compared to running each daemon individually in stand-alone mode. Primarily, inetd is used to spawn other daemons, but several trivial protocols are handled internally, such as chargen, auth, time, echo, discard, and daytime. This section covers the basics of configuring inetd. Configuration File Configuration of inetd is done by editing /etc/inetd.conf. Each line of this configuration file represents an application which can be started by inetd. By default, every line starts with a comment (#), meaning that inetd is not listening for any applications. To configure inetd to listen for an application's connections, remove the # at the beginning of the line for that application. After saving your edits, configure inetd to start at system boot by editing /etc/rc.conf: inetd_enable="YES" To start inetd now, so that it listens for the service you configured, type: &prompt.root; service inetd start Once inetd is started, it needs to be notified whenever a modification is made to /etc/inetd.conf: Reloading the <application>inetd</application> Configuration File &prompt.root; service inetd reload Typically, the default entry for an application does not need to be edited beyond removing the #. In some situations, it may be appropriate to edit the default entry. As an example, this is the default entry for &man.ftpd.8; over IPv4: ftp stream tcp nowait root /usr/libexec/ftpd ftpd -l The seven columns in an entry are as follows: service-name socket-type protocol {wait|nowait}[/max-child[/max-connections-per-ip-per-minute[/max-child-per-ip]]] user[:group][/login-class] server-program server-program-arguments where: service-name The service name of the daemon to start. It must correspond to a service listed in /etc/services. This determines which port inetd listens on for incoming connections to that service. When using a custom service, it must first be added to /etc/services. socket-type Either stream, dgram, raw, or seqpacket. Use stream for TCP connections and dgram for UDP services. protocol Use one of the following protocol names: Protocol Name Explanation tcp or tcp4 TCP IPv4 udp or udp4 UDP IPv4 tcp6 TCP IPv6 udp6 UDP IPv6 tcp46 Both TCP IPv4 and IPv6 udp46 Both UDP IPv4 and IPv6 {wait|nowait}[/max-child[/max-connections-per-ip-per-minute[/max-child-per-ip]]] In this field, or must be specified. , and are optional. indicates whether or not the service is able to handle its own socket. socket types must use while daemons, which are usually multi-threaded, should use . usually hands off multiple sockets to a single daemon, while spawns a child daemon for each new socket. The maximum number of child daemons inetd may spawn is set by . For example, to limit ten instances of the daemon, place a /10 after . Specifying /0 allows an unlimited number of children. limits the number of connections from any particular IP address per minute. Once the limit is reached, further connections from this IP address will be dropped until the end of the minute. For example, a value of /10 would limit any particular IP address to ten connection attempts per minute. limits the number of child processes that can be started on behalf on any single IP address at any moment. These options can limit excessive resource consumption and help to prevent Denial of Service attacks. An example can be seen in the default settings for &man.fingerd.8;: finger stream tcp nowait/3/10 nobody /usr/libexec/fingerd fingerd -k -s user The username the daemon will run as. Daemons typically run as root, daemon, or nobody. server-program The full path to the daemon. If the daemon is a service provided by inetd internally, use . server-program-arguments Used to specify any command arguments to be passed to the daemon on invocation. If the daemon is an internal service, use . Command-Line Options Like most server daemons, inetd has a number of options that can be used to modify its behavior. By default, inetd is started with -wW -C 60. These options enable TCP wrappers for all services, including internal services, and prevent any IP address from requesting any service more than 60 times per minute. To change the default options which are passed to inetd, add an entry for inetd_flags in /etc/rc.conf. If inetd is already running, restart it with service inetd restart. The available rate limiting options are: -c maximum Specify the default maximum number of simultaneous invocations of each service, where the default is unlimited. May be overridden on a per-service basis by using in /etc/inetd.conf. -C rate Specify the default maximum number of times a service can be invoked from a single IP address per minute. May be overridden on a per-service basis by using in /etc/inetd.conf. -R rate Specify the maximum number of times a service can be invoked in one minute, where the default is 256. A rate of 0 allows an unlimited number. -s maximum Specify the maximum number of times a service can be invoked from a single IP address at any one time, where the default is unlimited. May be overridden on a per-service basis by using in /etc/inetd.conf. Additional options are available. Refer to &man.inetd.8; for the full list of options. Security Considerations Many of the daemons which can be managed by inetd are not security-conscious. Some daemons, such as fingerd, can provide information that may be useful to an attacker. Only enable the services which are needed and monitor the system for excessive connection attempts. max-connections-per-ip-per-minute, max-child and max-child-per-ip can be used to limit such attacks. By default, TCP wrappers is enabled. Consult &man.hosts.access.5; for more information on placing TCP restrictions on various inetd invoked daemons. Network File System (NFS) Tom Rhodes Reorganized and enhanced by Bill Swingle Written by NFS &os; supports the Network File System (NFS), which allows a server to share directories and files with clients over a network. With NFS, users and programs can access files on remote systems as if they were stored locally. NFS has many practical uses. Some of the more common uses include: Data that would otherwise be duplicated on each client can be kept in a single location and accessed by clients on the network. Several clients may need access to the /usr/ports/distfiles directory. Sharing that directory allows for quick access to the source files without having to download them to each client. On large networks, it is often more convenient to configure a central NFS server on which all user home directories are stored. Users can log into a client anywhere on the network and have access to their home directories. Administration of NFS exports is simplified. For example, there is only one file system where security or backup policies must be set. Removable media storage devices can be used by other machines on the network. This reduces the number of devices throughout the network and provides a centralized location to manage their security. It is often more convenient to install software on multiple machines from a centralized installation media. NFS consists of a server and one or more clients. The client remotely accesses the data that is stored on the server machine. In order for this to function properly, a few processes have to be configured and running. These daemons must be running on the server: NFS server file server UNIX clients rpcbind mountd nfsd Daemon Description nfsd The NFS daemon which services requests from NFS clients. mountd The NFS mount daemon which carries out requests received from nfsd. rpcbind This daemon allows NFS clients to discover which port the NFS server is using. Running &man.nfsiod.8; on the client can improve performance, but is not required. Configuring the Server NFS configuration The file systems which the NFS server will share are specified in /etc/exports. Each line in this file specifies a file system to be exported, which clients have access to that file system, and any access options. When adding entries to this file, each exported file system, its properties, and allowed hosts must occur on a single line. If no clients are listed in the entry, then any client on the network can mount that file system. NFS export examples The following /etc/exports entries demonstrate how to export file systems. The examples can be modified to match the file systems and client names on the reader's network. There are many options that can be used in this file, but only a few will be mentioned here. See &man.exports.5; for the full list of options. This example shows how to export /cdrom to three hosts named alpha, bravo, and charlie: /cdrom -ro alpha bravo charlie The -ro flag makes the file system read-only, preventing clients from making any changes to the exported file system. This example assumes that the host names are either in DNS or in /etc/hosts. Refer to &man.hosts.5; if the network does not have a DNS server. The next example exports /home to three clients by IP address. This can be useful for networks without DNS or /etc/hosts entries. The -alldirs flag allows subdirectories to be mount points. In other words, it will not automatically mount the subdirectories, but will permit the client to mount the directories that are required as needed. /usr/home -alldirs 10.0.0.2 10.0.0.3 10.0.0.4 This next example exports /a so that two clients from different domains may access that file system. The allows root on the remote system to write data on the exported file system as root. If -maproot=root is not specified, the client's root user will be mapped to the server's nobody account and will be subject to the access limitations defined for nobody. /a -maproot=root host.example.com box.example.org A client can only be specified once per file system. For example, if /usr is a single file system, these entries would be invalid as both entries specify the same host: # Invalid when /usr is one file system /usr/src client /usr/ports client The correct format for this situation is to use one entry: /usr/src /usr/ports client The following is an example of a valid export list, where /usr and /exports are local file systems: # Export src and ports to client01 and client02, but only # client01 has root privileges on it /usr/src /usr/ports -maproot=root client01 /usr/src /usr/ports client02 # The client machines have root and can mount anywhere # on /exports. Anyone in the world can mount /exports/obj read-only /exports -alldirs -maproot=root client01 client02 /exports/obj -ro To enable the processes required by the NFS server at boot time, add these options to /etc/rc.conf: rpcbind_enable="YES" nfs_server_enable="YES" mountd_flags="-r" The server can be started now by running this command: &prompt.root; service nfsd start Whenever the NFS server is started, mountd also starts automatically. However, mountd only reads /etc/exports when it is started. To make subsequent /etc/exports edits take effect immediately, force mountd to reread it: &prompt.root; service mountd reload Configuring the Client To enable NFS clients, set this option in each client's /etc/rc.conf: nfs_client_enable="YES" Then, run this command on each NFS client: &prompt.root; service nfsclient start The client now has everything it needs to mount a remote file system. In these examples, the server's name is server and the client's name is client. To mount /home on server to the /mnt mount point on client: NFS mounting &prompt.root; mount server:/home /mnt The files and directories in /home will now be available on client, in the /mnt directory. To mount a remote file system each time the client boots, add it to /etc/fstab: server:/home /mnt nfs rw 0 0 Refer to &man.fstab.5; for a description of all available options. Locking Some applications require file locking to operate correctly. To enable locking, add these lines to /etc/rc.conf on both the client and server: rpc_lockd_enable="YES" rpc_statd_enable="YES" Then start the applications: &prompt.root; service lockd start &prompt.root; service statd start If locking is not required on the server, the NFS client can be configured to lock locally by including when running mount. Refer to &man.mount.nfs.8; for further details. Automating Mounts with &man.amd.8; Wylie Stilwell Contributed by Chern Lee Rewritten by amd automatic mounter daemon The automatic mounter daemon, amd, automatically mounts a remote file system whenever a file or directory within that file system is accessed. File systems that are inactive for a period of time will be automatically unmounted by amd. This daemon provides an alternative to modifying /etc/fstab to list every client. It operates by attaching itself as an NFS server to the /host and /net directories. When a file is accessed within one of these directories, amd looks up the corresponding remote mount and automatically mounts it. /net is used to mount an exported file system from an IP address while /host is used to mount an export from a remote hostname. For instance, an attempt to access a file within /host/foobar/usr would tell amd to mount the /usr export on the host foobar. Mounting an Export with <application>amd</application> In this example, showmount -e shows the exported file systems that can be mounted from the NFS server, foobar: &prompt.user; showmount -e foobar Exports list on foobar: /usr 10.10.10.0 /a 10.10.10.0 &prompt.user; cd /host/foobar/usr The output from showmount shows /usr as an export. When changing directories to /host/foobar/usr, amd intercepts the request and attempts to resolve the hostname foobar. If successful, amd automatically mounts the desired export. To enable amd at boot time, add this line to /etc/rc.conf: amd_enable="YES" To start amd now: &prompt.root; service amd start Custom flags can be passed to amd from the amd_flags environment variable. By default, amd_flags is set to: amd_flags="-a /.amd_mnt -l syslog /host /etc/amd.map /net /etc/amd.map" The default options with which exports are mounted are defined in /etc/amd.map. Some of the more advanced features of amd are defined in /etc/amd.conf. Consult &man.amd.8; and &man.amd.conf.5; for more information. Automating Mounts with &man.autofs.5; The &man.autofs.5; automount facility is supported starting with &os; 10.1-RELEASE. To use the automounter functionality in older versions of &os;, use &man.amd.8; instead. This chapter only describes the &man.autofs.5; automounter. autofs automounter subsystem The &man.autofs.5; facility is a common name for several components that, together, allow for automatic mounting of remote and local filesystems whenever a file or directory within that file system is accessed. It consists of the kernel component, &man.autofs.5;, and several userspace applications: &man.automount.8;, &man.automountd.8; and &man.autounmountd.8;. It serves as an alternative for &man.amd.8; from previous &os; releases. Amd is still provided for backward compatibility purposes, as the two use different map format; the one used by autofs is the same as with other SVR4 automounters, such as the ones in Solaris, MacOS X, and Linux. The &man.autofs.5; virtual filesystem is mounted on specified mountpoints by &man.automount.8;, usually invoked during boot. Whenever a process attempts to access file within the &man.autofs.5; mountpoint, the kernel will notify &man.automountd.8; daemon and pause the triggering process. The &man.automountd.8; daemon will handle kernel requests by finding the proper map and mounting the filesystem according to it, then signal the kernel to release blocked process. The &man.autounmountd.8; daemon automatically unmounts automounted filesystems after some time, unless they are still being used. The primary autofs configuration file is /etc/auto_master. It assigns individual maps to top-level mounts. For an explanation of auto_master and the map syntax, refer to &man.auto.master.5;. There is a special automounter map mounted on /net. When a file is accessed within this directory, &man.autofs.5; looks up the corresponding remote mount and automatically mounts it. For instance, an attempt to access a file within /net/foobar/usr would tell &man.automountd.8; to mount the /usr export from the host foobar. Mounting an Export with &man.autofs.5; In this example, showmount -e shows the exported file systems that can be mounted from the NFS server, foobar: &prompt.user; showmount -e foobar Exports list on foobar: /usr 10.10.10.0 /a 10.10.10.0 &prompt.user; cd /net/foobar/usr The output from showmount shows /usr as an export. When changing directories to /host/foobar/usr, &man.automountd.8; intercepts the request and attempts to resolve the hostname foobar. If successful, &man.automountd.8; automatically mounts the source export. To enable &man.autofs.5; at boot time, add this line to /etc/rc.conf: autofs_enable="YES" Then &man.autofs.5; can be started by running: &prompt.root; service automount start &prompt.root; service automountd start &prompt.root; service autounmountd start The &man.autofs.5; map format is the same as in other operating systems. Information about this format from other sources can be useful, like the Mac OS X document. Consult the &man.automount.8;, &man.automountd.8;, &man.autounmountd.8;, and &man.auto.master.5; manual pages for more information. Network Information System (<acronym>NIS</acronym>) NIS Solaris HP-UX AIX Linux NetBSD OpenBSD yellow pages NIS Network Information System (NIS) is designed to centralize administration of &unix;-like systems such as &solaris;, HP-UX, &aix;, Linux, NetBSD, OpenBSD, and &os;. NIS was originally known as Yellow Pages but the name was changed due to trademark issues. This is the reason why NIS commands begin with yp. NIS domains NIS is a Remote Procedure Call (RPC)-based client/server system that allows a group of machines within an NIS domain to share a common set of configuration files. This permits a system administrator to set up NIS client systems with only minimal configuration data and to add, remove, or modify configuration data from a single location. &os; uses version 2 of the NIS protocol. <acronym>NIS</acronym> Terms and Processes Table 28.1 summarizes the terms and important processes used by NIS: rpcbind portmap <acronym>NIS</acronym> Terminology Term Description NIS domain name NIS servers and clients share an NIS domain name. Typically, this name does not have anything to do with DNS. &man.rpcbind.8; This service enables RPC and must be running in order to run an NIS server or act as an NIS client. &man.ypbind.8; This service binds an NIS client to its NIS server. It will take the NIS domain name and use RPC to connect to the server. It is the core of client/server communication in an NIS environment. If this service is not running on a client machine, it will not be able to access the NIS server. &man.ypserv.8; This is the process for the NIS server. If this service stops running, the server will no longer be able to respond to NIS requests so hopefully, there is a slave server to take over. Some non-&os; clients will not try to reconnect using a slave server and the ypbind process may need to be restarted on these clients. &man.rpc.yppasswdd.8; This process only runs on NIS master servers. This daemon allows NIS clients to change their NIS passwords. If this daemon is not running, users will have to login to the NIS master server and change their passwords there.
Machine Types NIS master server NIS slave server NIS client There are three types of hosts in an NIS environment: NIS master server This server acts as a central repository for host configuration information and maintains the authoritative copy of the files used by all of the NIS clients. The passwd, group, and other various files used by NIS clients are stored on the master server. While it is possible for one machine to be an NIS master server for more than one NIS domain, this type of configuration will not be covered in this chapter as it assumes a relatively small-scale NIS environment. NIS slave servers NIS slave servers maintain copies of the NIS master's data files in order to provide redundancy. Slave servers also help to balance the load of the master server as NIS clients always attach to the NIS server which responds first. NIS clients NIS clients authenticate against the NIS server during log on. Information in many files can be shared using NIS. The master.passwd, group, and hosts files are commonly shared via NIS. Whenever a process on a client needs information that would normally be found in these files locally, it makes a query to the NIS server that it is bound to instead. Planning Considerations This section describes a sample NIS environment which consists of 15 &os; machines with no centralized point of administration. Each machine has its own /etc/passwd and /etc/master.passwd. These files are kept in sync with each other only through manual intervention. Currently, when a user is added to the lab, the process must be repeated on all 15 machines. The configuration of the lab will be as follows: Machine name IP address Machine role ellington 10.0.0.2 NIS master coltrane 10.0.0.3 NIS slave basie 10.0.0.4 Faculty workstation bird 10.0.0.5 Client machine cli[1-11] 10.0.0.[6-17] Other client machines If this is the first time an NIS scheme is being developed, it should be thoroughly planned ahead of time. Regardless of network size, several decisions need to be made as part of the planning process. Choosing a <acronym>NIS</acronym> Domain Name NIS domain name When a client broadcasts its requests for info, it includes the name of the NIS domain that it is part of. This is how multiple servers on one network can tell which server should answer which request. Think of the NIS domain name as the name for a group of hosts. Some organizations choose to use their Internet domain name for their NIS domain name. This is not recommended as it can cause confusion when trying to debug network problems. The NIS domain name should be unique within the network and it is helpful if it describes the group of machines it represents. For example, the Art department at Acme Inc. might be in the acme-art NIS domain. This example will use the domain name test-domain. However, some non-&os; operating systems require the NIS domain name to be the same as the Internet domain name. If one or more machines on the network have this restriction, the Internet domain name must be used as the NIS domain name. Physical Server Requirements There are several things to keep in mind when choosing a machine to use as a NIS server. Since NIS clients depend upon the availability of the server, choose a machine that is not rebooted frequently. The NIS server should ideally be a stand alone machine whose sole purpose is to be an NIS server. If the network is not heavily used, it is acceptable to put the NIS server on a machine running other services. However, if the NIS server becomes unavailable, it will adversely affect all NIS clients. Configuring the <acronym>NIS</acronym> Master Server The canonical copies of all NIS files are stored on the master server. The databases used to store the information are called NIS maps. In &os;, these maps are stored in /var/yp/[domainname] where [domainname] is the name of the NIS domain. Since multiple domains are supported, it is possible to have several directories, one for each domain. Each domain will have its own independent set of maps. NIS master and slave servers handle all NIS requests through &man.ypserv.8;. This daemon is responsible for receiving incoming requests from NIS clients, translating the requested domain and map name to a path to the corresponding database file, and transmitting data from the database back to the client. NIS server configuration Setting up a master NIS server can be relatively straight forward, depending on environmental needs. Since &os; provides built-in NIS support, it only needs to be enabled by adding the following lines to /etc/rc.conf: nisdomainname="test-domain" nis_server_enable="YES" nis_yppasswdd_enable="YES" This line sets the NIS domain name to test-domain. This automates the start up of the NIS server processes when the system boots. This enables the &man.rpc.yppasswdd.8; daemon so that users can change their NIS password from a client machine. Care must be taken in a multi-server domain where the server machines are also NIS clients. It is generally a good idea to force the servers to bind to themselves rather than allowing them to broadcast bind requests and possibly become bound to each other. Strange failure modes can result if one server goes down and others are dependent upon it. Eventually, all the clients will time out and attempt to bind to other servers, but the delay involved can be considerable and the failure mode is still present since the servers might bind to each other all over again. A server that is also a client can be forced to bind to a particular server by adding these additional lines to /etc/rc.conf: nis_client_enable="YES" # run client stuff as well nis_client_flags="-S NIS domain,server" After saving the edits, type /etc/netstart to restart the network and apply the values defined in /etc/rc.conf. Before initializing the NIS maps, start &man.ypserv.8;: &prompt.root; service ypserv start Initializing the <acronym>NIS</acronym> Maps NIS maps NIS maps are generated from the configuration files in /etc on the NIS master, with one exception: /etc/master.passwd. This is to prevent the propagation of passwords to all the servers in the NIS domain. Therefore, before the NIS maps are initialized, configure the primary password files: &prompt.root; cp /etc/master.passwd /var/yp/master.passwd &prompt.root; cd /var/yp &prompt.root; vi master.passwd It is advisable to remove all entries for system accounts as well as any user accounts that do not need to be propagated to the NIS clients, such as the root and any other administrative accounts. Ensure that the /var/yp/master.passwd is neither group or world readable by setting its permissions to 600. After completing this task, initialize the NIS maps. &os; includes the &man.ypinit.8; script to do this. When generating maps for the master server, include and specify the NIS domain name: ellington&prompt.root; ypinit -m test-domain Server Type: MASTER Domain: test-domain Creating an YP server will require that you answer a few questions. Questions will all be asked at the beginning of the procedure. Do you want this procedure to quit on non-fatal errors? [y/n: n] n Ok, please remember to go back and redo manually whatever fails. If not, something might not work. At this point, we have to construct a list of this domains YP servers. rod.darktech.org is already known as master server. Please continue to add any slave servers, one per line. When you are done with the list, type a <control D>. master server : ellington next host to add: coltrane next host to add: ^D The current list of NIS servers looks like this: ellington coltrane Is this correct? [y/n: y] y [..output from map generation..] NIS Map update completed. ellington has been setup as an YP master server without any errors. This will create /var/yp/Makefile from /var/yp/Makefile.dist. By default, this file assumes that the environment has a single NIS server with only &os; clients. Since test-domain has a slave server, edit this line in /var/yp/Makefile so that it begins with a comment (#): NOPUSH = "True" Adding New Users Every time a new user is created, the user account must be added to the master NIS server and the NIS maps rebuilt. Until this occurs, the new user will not be able to login anywhere except on the NIS master. For example, to add the new user jsmith to the test-domain domain, run these commands on the master server: &prompt.root; pw useradd jsmith &prompt.root; cd /var/yp &prompt.root; make test-domain The user could also be added using adduser jsmith instead of pw useradd smith. Setting up a <acronym>NIS</acronym> Slave Server NIS slave server To set up an NIS slave server, log on to the slave server and edit /etc/rc.conf as for the master server. Do not generate any NIS maps, as these already exist on the master server. When running ypinit on the slave server, use (for slave) instead of (for master). This option requires the name of the NIS master in addition to the domain name, as seen in this example: coltrane&prompt.root; ypinit -s ellington test-domain Server Type: SLAVE Domain: test-domain Master: ellington Creating an YP server will require that you answer a few questions. Questions will all be asked at the beginning of the procedure. Do you want this procedure to quit on non-fatal errors? [y/n: n] n Ok, please remember to go back and redo manually whatever fails. If not, something might not work. There will be no further questions. The remainder of the procedure should take a few minutes, to copy the databases from ellington. Transferring netgroup... ypxfr: Exiting: Map successfully transferred Transferring netgroup.byuser... ypxfr: Exiting: Map successfully transferred Transferring netgroup.byhost... ypxfr: Exiting: Map successfully transferred Transferring master.passwd.byuid... ypxfr: Exiting: Map successfully transferred Transferring passwd.byuid... ypxfr: Exiting: Map successfully transferred Transferring passwd.byname... ypxfr: Exiting: Map successfully transferred Transferring group.bygid... ypxfr: Exiting: Map successfully transferred Transferring group.byname... ypxfr: Exiting: Map successfully transferred Transferring services.byname... ypxfr: Exiting: Map successfully transferred Transferring rpc.bynumber... ypxfr: Exiting: Map successfully transferred Transferring rpc.byname... ypxfr: Exiting: Map successfully transferred Transferring protocols.byname... ypxfr: Exiting: Map successfully transferred Transferring master.passwd.byname... ypxfr: Exiting: Map successfully transferred Transferring networks.byname... ypxfr: Exiting: Map successfully transferred Transferring networks.byaddr... ypxfr: Exiting: Map successfully transferred Transferring netid.byname... ypxfr: Exiting: Map successfully transferred Transferring hosts.byaddr... ypxfr: Exiting: Map successfully transferred Transferring protocols.bynumber... ypxfr: Exiting: Map successfully transferred Transferring ypservers... ypxfr: Exiting: Map successfully transferred Transferring hosts.byname... ypxfr: Exiting: Map successfully transferred coltrane has been setup as an YP slave server without any errors. Remember to update map ypservers on ellington. This will generate a directory on the slave server called /var/yp/test-domain which contains copies of the NIS master server's maps. Adding these /etc/crontab entries on each slave server will force the slaves to sync their maps with the maps on the master server: 20 * * * * root /usr/libexec/ypxfr passwd.byname 21 * * * * root /usr/libexec/ypxfr passwd.byuid These entries are not mandatory because the master server automatically attempts to push any map changes to its slaves. However, since clients may depend upon the slave server to provide correct password information, it is recommended to force frequent password map updates. This is especially important on busy networks where map updates might not always complete. To finish the configuration, run /etc/netstart on the slave server in order to start the NIS services. Setting Up an <acronym>NIS</acronym> Client An NIS client binds to an NIS server using &man.ypbind.8;. This daemon broadcasts RPC requests on the local network. These requests specify the domain name configured on the client. If an NIS server in the same domain receives one of the broadcasts, it will respond to ypbind, which will record the server's address. If there are several servers available, the client will use the address of the first server to respond and will direct all of its NIS requests to that server. The client will automatically ping the server on a regular basis to make sure it is still available. If it fails to receive a reply within a reasonable amount of time, ypbind will mark the domain as unbound and begin broadcasting again in the hopes of locating another server. NIS client configuration To configure a &os; machine to be an NIS client: Edit /etc/rc.conf and add the following lines in order to set the NIS domain name and start &man.ypbind.8; during network startup: nisdomainname="test-domain" nis_client_enable="YES" To import all possible password entries from the NIS server, use vipw to remove all user accounts except one from /etc/master.passwd. When removing the accounts, keep in mind that at least one local account should remain and this account should be a member of wheel. If there is a problem with NIS, this local account can be used to log in remotely, become the superuser, and fix the problem. Before saving the edits, add the following line to the end of the file: +::::::::: This line configures the client to provide anyone with a valid account in the NIS server's password maps an account on the client. There are many ways to configure the NIS client by modifying this line. One method is described in . For more detailed reading, refer to the book Managing NFS and NIS, published by O'Reilly Media. To import all possible group entries from the NIS server, add this line to /etc/group: +:*:: To start the NIS client immediately, execute the following commands as the superuser: &prompt.root; /etc/netstart &prompt.root; service ypbind start After completing these steps, running ypcat passwd on the client should show the server's passwd map. <acronym>NIS</acronym> Security Since RPC is a broadcast-based service, any system running ypbind within the same domain can retrieve the contents of the NIS maps. To prevent unauthorized transactions, &man.ypserv.8; supports a feature called securenets which can be used to restrict access to a given set of hosts. By default, this information is stored in /var/yp/securenets, unless &man.ypserv.8; is started with and an alternate path. This file contains entries that consist of a network specification and a network mask separated by white space. Lines starting with # are considered to be comments. A sample securenets might look like this: # allow connections from local host -- mandatory 127.0.0.1 255.255.255.255 # allow connections from any host # on the 192.168.128.0 network 192.168.128.0 255.255.255.0 # allow connections from any host # between 10.0.0.0 to 10.0.15.255 # this includes the machines in the testlab 10.0.0.0 255.255.240.0 If &man.ypserv.8; receives a request from an address that matches one of these rules, it will process the request normally. If the address fails to match a rule, the request will be ignored and a warning message will be logged. If the securenets does not exist, ypserv will allow connections from any host. is an alternate mechanism for providing access control instead of securenets. While either access control mechanism adds some security, they are both vulnerable to IP spoofing attacks. All NIS-related traffic should be blocked at the firewall. Servers using securenets may fail to serve legitimate NIS clients with archaic TCP/IP implementations. Some of these implementations set all host bits to zero when doing broadcasts or fail to observe the subnet mask when calculating the broadcast address. While some of these problems can be fixed by changing the client configuration, other problems may force the retirement of these client systems or the abandonment of securenets. TCP Wrapper The use of TCP Wrapper increases the latency of the NIS server. The additional delay may be long enough to cause timeouts in client programs, especially in busy networks with slow NIS servers. If one or more clients suffer from latency, convert those clients into NIS slave servers and force them to bind to themselves. Barring Some Users In this example, the basie system is a faculty workstation within the NIS domain. The passwd map on the master NIS server contains accounts for both faculty and students. This section demonstrates how to allow faculty logins on this system while refusing student logins. To prevent specified users from logging on to a system, even if they are present in the NIS database, use vipw to add -username with the correct number of colons towards the end of /etc/master.passwd on the client, where username is the username of a user to bar from logging in. The line with the blocked user must be before the + line that allows NIS users. In this example, bill is barred from logging on to basie: basie&prompt.root; cat /etc/master.passwd root:[password]:0:0::0:0:The super-user:/root:/bin/csh toor:[password]:0:0::0:0:The other super-user:/root:/bin/sh daemon:*:1:1::0:0:Owner of many system processes:/root:/sbin/nologin operator:*:2:5::0:0:System &:/:/sbin/nologin bin:*:3:7::0:0:Binaries Commands and Source,,,:/:/sbin/nologin tty:*:4:65533::0:0:Tty Sandbox:/:/sbin/nologin kmem:*:5:65533::0:0:KMem Sandbox:/:/sbin/nologin games:*:7:13::0:0:Games pseudo-user:/usr/games:/sbin/nologin news:*:8:8::0:0:News Subsystem:/:/sbin/nologin man:*:9:9::0:0:Mister Man Pages:/usr/share/man:/sbin/nologin bind:*:53:53::0:0:Bind Sandbox:/:/sbin/nologin uucp:*:66:66::0:0:UUCP pseudo-user:/var/spool/uucppublic:/usr/libexec/uucp/uucico xten:*:67:67::0:0:X-10 daemon:/usr/local/xten:/sbin/nologin pop:*:68:6::0:0:Post Office Owner:/nonexistent:/sbin/nologin nobody:*:65534:65534::0:0:Unprivileged user:/nonexistent:/sbin/nologin -bill::::::::: +::::::::: basie&prompt.root; Using Netgroups netgroups Barring specified users from logging on to individual systems becomes unscaleable on larger networks and quickly loses the main benefit of NIS: centralized administration. Netgroups were developed to handle large, complex networks with hundreds of users and machines. Their use is comparable to &unix; groups, where the main difference is the lack of a numeric ID and the ability to define a netgroup by including both user accounts and other netgroups. To expand on the example used in this chapter, the NIS domain will be extended to add the users and systems shown in Tables 28.2 and 28.3: Additional Users User Name(s) Description alpha, beta IT department employees charlie, delta IT department apprentices echo, foxtrott, golf, ... employees able, baker, ... interns
Additional Systems Machine Name(s) Description war, death, famine, pollution Only IT employees are allowed to log onto these servers. pride, greed, envy, wrath, lust, sloth All members of the IT department are allowed to login onto these servers. one, two, three, four, ... Ordinary workstations used by employees. trashcan A very old machine without any critical data. Even interns are allowed to use this system.
When using netgroups to configure this scenario, each user is assigned to one or more netgroups and logins are then allowed or forbidden for all members of the netgroup. When adding a new machine, login restrictions must be defined for all netgroups. When a new user is added, the account must be added to one or more netgroups. If the NIS setup is planned carefully, only one central configuration file needs modification to grant or deny access to machines. The first step is the initialization of the NIS netgroup map. In &os;, this map is not created by default. On the NIS master server, use an editor to create a map named /var/yp/netgroup. This example creates four netgroups to represent IT employees, IT apprentices, employees, and interns: IT_EMP (,alpha,test-domain) (,beta,test-domain) IT_APP (,charlie,test-domain) (,delta,test-domain) USERS (,echo,test-domain) (,foxtrott,test-domain) \ (,golf,test-domain) INTERNS (,able,test-domain) (,baker,test-domain) Each entry configures a netgroup. The first column in an entry is the name of the netgroup. Each set of brackets represents either a group of one or more users or the name of another netgroup. When specifying a user, the three comma-delimited fields inside each group represent: The name of the host(s) where the other fields representing the user are valid. If a hostname is not specified, the entry is valid on all hosts. The name of the account that belongs to this netgroup. The NIS domain for the account. Accounts may be imported from other NIS domains into a netgroup. If a group contains multiple users, separate each user with whitespace. Additionally, each field may contain wildcards. See &man.netgroup.5; for details. netgroups Netgroup names longer than 8 characters should not be used. The names are case sensitive and using capital letters for netgroup names is an easy way to distinguish between user, machine and netgroup names. Some non-&os; NIS clients cannot handle netgroups containing more than 15 entries. This limit may be circumvented by creating several sub-netgroups with 15 users or fewer and a real netgroup consisting of the sub-netgroups, as seen in this example: BIGGRP1 (,joe1,domain) (,joe2,domain) (,joe3,domain) [...] BIGGRP2 (,joe16,domain) (,joe17,domain) [...] BIGGRP3 (,joe31,domain) (,joe32,domain) BIGGROUP BIGGRP1 BIGGRP2 BIGGRP3 Repeat this process if more than 225 (15 times 15) users exist within a single netgroup. To activate and distribute the new NIS map: ellington&prompt.root; cd /var/yp ellington&prompt.root; make This will generate the three NIS maps netgroup, netgroup.byhost and netgroup.byuser. Use the map key option of &man.ypcat.1; to check if the new NIS maps are available: ellington&prompt.user; ypcat -k netgroup ellington&prompt.user; ypcat -k netgroup.byhost ellington&prompt.user; ypcat -k netgroup.byuser The output of the first command should resemble the contents of /var/yp/netgroup. The second command only produces output if host-specific netgroups were created. The third command is used to get the list of netgroups for a user. To configure a client, use &man.vipw.8; to specify the name of the netgroup. For example, on the server named war, replace this line: +::::::::: with +@IT_EMP::::::::: This specifies that only the users defined in the netgroup IT_EMP will be imported into this system's password database and only those users are allowed to login to this system. This configuration also applies to the ~ function of the shell and all routines which convert between user names and numerical user IDs. In other words, cd ~user will not work, ls -l will show the numerical ID instead of the username, and find . -user joe -print will fail with the message No such user. To fix this, import all user entries without allowing them to login into the servers. This can be achieved by adding an extra line: +:::::::::/sbin/nologin This line configures the client to import all entries but to replace the shell in those entries with /sbin/nologin. Make sure that extra line is placed after +@IT_EMP:::::::::. Otherwise, all user accounts imported from NIS will have /sbin/nologin as their login shell and no one will be able to login to the system. To configure the less important servers, replace the old +::::::::: on the servers with these lines: +@IT_EMP::::::::: +@IT_APP::::::::: +:::::::::/sbin/nologin The corresponding lines for the workstations would be: +@IT_EMP::::::::: +@USERS::::::::: +:::::::::/sbin/nologin NIS supports the creation of netgroups from other netgroups which can be useful if the policy regarding user access changes. One possibility is the creation of role-based netgroups. For example, one might create a netgroup called BIGSRV to define the login restrictions for the important servers, another netgroup called SMALLSRV for the less important servers, and a third netgroup called USERBOX for the workstations. Each of these netgroups contains the netgroups that are allowed to login onto these machines. The new entries for the NIS netgroup map would look like this: BIGSRV IT_EMP IT_APP SMALLSRV IT_EMP IT_APP ITINTERN USERBOX IT_EMP ITINTERN USERS This method of defining login restrictions works reasonably well when it is possible to define groups of machines with identical restrictions. Unfortunately, this is the exception and not the rule. Most of the time, the ability to define login restrictions on a per-machine basis is required. Machine-specific netgroup definitions are another possibility to deal with the policy changes. In this scenario, the /etc/master.passwd of each system contains two lines starting with +. The first line adds a netgroup with the accounts allowed to login onto this machine and the second line adds all other accounts with /sbin/nologin as shell. It is recommended to use the ALL-CAPS version of the hostname as the name of the netgroup: +@BOXNAME::::::::: +:::::::::/sbin/nologin Once this task is completed on all the machines, there is no longer a need to modify the local versions of /etc/master.passwd ever again. All further changes can be handled by modifying the NIS map. Here is an example of a possible netgroup map for this scenario: # Define groups of users first IT_EMP (,alpha,test-domain) (,beta,test-domain) IT_APP (,charlie,test-domain) (,delta,test-domain) DEPT1 (,echo,test-domain) (,foxtrott,test-domain) DEPT2 (,golf,test-domain) (,hotel,test-domain) DEPT3 (,india,test-domain) (,juliet,test-domain) ITINTERN (,kilo,test-domain) (,lima,test-domain) D_INTERNS (,able,test-domain) (,baker,test-domain) # # Now, define some groups based on roles USERS DEPT1 DEPT2 DEPT3 BIGSRV IT_EMP IT_APP SMALLSRV IT_EMP IT_APP ITINTERN USERBOX IT_EMP ITINTERN USERS # # And a groups for a special tasks # Allow echo and golf to access our anti-virus-machine SECURITY IT_EMP (,echo,test-domain) (,golf,test-domain) # # machine-based netgroups # Our main servers WAR BIGSRV FAMINE BIGSRV # User india needs access to this server POLLUTION BIGSRV (,india,test-domain) # # This one is really important and needs more access restrictions DEATH IT_EMP # # The anti-virus-machine mentioned above ONE SECURITY # # Restrict a machine to a single user TWO (,hotel,test-domain) # [...more groups to follow] It may not always be advisable to use machine-based netgroups. When deploying a couple of dozen or hundreds of systems, role-based netgroups instead of machine-based netgroups may be used to keep the size of the NIS map within reasonable limits. Password Formats NIS password formats NIS requires that all hosts within an NIS domain use the same format for encrypting passwords. If users have trouble authenticating on an NIS client, it may be due to a differing password format. In a heterogeneous network, the format must be supported by all operating systems, where DES is the lowest common standard. To check which format a server or client is using, look at this section of /etc/login.conf: default:\ :passwd_format=des:\ :copyright=/etc/COPYRIGHT:\ [Further entries elided] In this example, the system is using the DES format. Other possible values are blf for Blowfish and md5 for MD5 encrypted passwords. If the format on a host needs to be edited to match the one being used in the NIS domain, the login capability database must be rebuilt after saving the change: &prompt.root; cap_mkdb /etc/login.conf The format of passwords for existing user accounts will not be updated until each user changes their password after the login capability database is rebuilt. Lightweight Directory Access Protocol (<acronym>LDAP</acronym>) Tom Rhodes Written by LDAP The Lightweight Directory Access Protocol (LDAP) is an application layer protocol used to access, modify, and authenticate objects using a distributed directory information service. Think of it as a phone or record book which stores several levels of hierarchical, homogeneous information. It is used in Active Directory and OpenLDAP networks and allows users to access to several levels of internal information utilizing a single account. For example, email authentication, pulling employee contact information, and internal website authentication might all make use of a single user account in the LDAP server's record base. This section provides a quick start guide for configuring an LDAP server on a &os; system. It assumes that the administrator already has a design plan which includes the type of information to store, what that information will be used for, which users should have access to that information, and how to secure this information from unauthorized access. <acronym>LDAP</acronym> Terminology and Structure LDAP uses several terms which should be understood before starting the configuration. All directory entries consist of a group of attributes. Each of these attribute sets contains a unique identifier known as a Distinguished Name (DN) which is normally built from several other attributes such as the common or Relative Distinguished Name (RDN). Similar to how directories have absolute and relative paths, consider a DN as an absolute path and the RDN as the relative path. An example LDAP entry looks like the following. This example searches for the entry for the specified user account (uid), organizational unit (ou), and organization (o): &prompt.user; ldapsearch -xb "uid=trhodes,ou=users,o=example.com" # extended LDIF # # LDAPv3 # base <uid=trhodes,ou=users,o=example.com> with scope subtree # filter: (objectclass=*) # requesting: ALL # # trhodes, users, example.com dn: uid=trhodes,ou=users,o=example.com mail: trhodes@example.com cn: Tom Rhodes uid: trhodes telephoneNumber: (123) 456-7890 # search result search: 2 result: 0 Success # numResponses: 2 # numEntries: 1 This example entry shows the values for the dn, mail, cn, uid, and telephoneNumber attributes. The cn attribute is the RDN. More information about LDAP and its terminology can be found at http://www.openldap.org/doc/admin24/intro.html. Configuring an <acronym>LDAP</acronym> Server LDAP Server &os; does not provide a built-in LDAP server. Begin the configuration by installing the net/openldap24-server package or port. Since the port has many configurable options, it is recommended that the default options are reviewed to see if the package is sufficient, and to instead compile the port if any options should be changed. In most cases, the defaults are fine. However, if SQL support is needed, this option must be enabled and the port compiled using the instructions in . Next, create the directories to hold the data and to store the certificates: &prompt.root; mkdir /var/db/openldap-data &prompt.root; mkdir /usr/local/etc/openldap/private Copy over the database configuration file: &prompt.root; cp /usr/local/etc/openldap/DB_CONFIG.example /var/db/openldap-data/DB_CONFIG The next phase is to configure the certificate authority. The following commands must be executed from /usr/local/etc/openldap/private. This is important as the file permissions need to be restrictive and users should not have access to these files. To create the certificate authority, start with this command and follow the prompts: &prompt.root; openssl req -days 365 -nodes -new -x509 -keyout ca.key -out ../ca.crt The entries for the prompts may be generic except for the Common Name. This entry must be different than the system hostname. If this will be a self signed certificate, prefix the hostname with CA for certificate authority. The next task is to create a certificate signing request and a private key. Input this command and follow the prompts: &prompt.root; openssl req -days 365 -nodes -new -keyout server.key -out server.csr During the certificate generation process, be sure to correctly set the Common Name attribute. Once complete, sign the key: &prompt.root; openssl x509 -req -days 365 -in server.csr -out ../server.crt -CA ../ca.crt -CAkey ca.key -CAcreateserial The final part of the certificate generation process is to generate and sign the client certificates: &prompt.root; openssl req -days 365 -nodes -new -keyout client.key -out client.csr &prompt.root; openssl x509 -req -days 3650 -in client.csr -out ../client.crt -CA ../ca.crt -CAkey ca.key Remember to use the same Common Name attribute when prompted. When finished, ensure that a total of eight (8) new files have been generated through the proceeding commands. If so, the next step is to edit /usr/local/etc/openldap/slapd.conf and add the following options: TLSCipherSuite HIGH:MEDIUM:+SSLv3 TLSCertificateFile /usr/local/etc/openldap/server.crt TLSCertificateKeyFile /usr/local/etc/openldap/private/server.key TLSCACertificateFile /usr/local/etc/openldap/ca.crt Then, edit /usr/local/etc/openldap/ldap.conf and add the following lines: TLS_CACERT /usr/local/etc/openldap/ca.crt TLS_CIPHER_SUITE HIGH:MEDIUM:+SSLv3 While editing this file, uncomment the following entries and set them to the desired values: , , and . Set the to contain and . Then, add two entries pointing to the certificate authority. When finished, the entries should look similar to the following: BASE dc=example,dc=com URI ldap:// ldaps:// SIZELIMIT 12 TIMELIMIT 15 TLS_CACERT /usr/local/etc/openldap/ca.crt TLS_CIPHER_SUITE HIGH:MEDIUM:+SSLv3 The default password for the server should then be changed: &prompt.root; slappasswd -h "{SHA}" >> /usr/local/etc/openldap/slapd.conf This command will prompt for the password and, if the process does not fail, a password hash will be added to the end of slapd.conf. Several hashing formats are supported. Refer to the manual page for slappasswd for more information. Next, edit /usr/local/etc/openldap/slapd.conf and add the following lines: password-hash {sha} allow bind_v2 The in this file must be updated to match the used in /usr/local/etc/openldap/ldap.conf and should also be set. A recommended value for is something like . Before saving this file, place the in front of the password output from slappasswd and delete the old . The end result should look similar to this: TLSCipherSuite HIGH:MEDIUM:+SSLv3 TLSCertificateFile /usr/local/etc/openldap/server.crt TLSCertificateKeyFile /usr/local/etc/openldap/private/server.key TLSCACertificateFile /usr/local/etc/openldap/ca.crt rootpw {SHA}W6ph5Mm5Pz8GgiULbPgzG37mj9g= Finally, enable the OpenLDAP service in /etc/rc.conf and set the URI: slapd_enable="YES" slapd_flags="-4 -h ldaps:///" At this point the server can be started and tested: &prompt.root; service slapd start If everything is configured correctly, a search of the directory should show a successful connection with a single response as in this example: &prompt.root; ldapsearch -Z # extended LDIF # # LDAPv3 # base <dc=example,dc=com> (default) with scope subtree # filter: (objectclass=*) # requesting: ALL # # search result search: 3 result: 32 No such object # numResponses: 1 If the command fails and the configuration looks correct, stop the slapd service and restart it with debugging options: &prompt.root; service slapd stop &prompt.root; /usr/local/libexec/slapd -d -1 Once the service is responding, the directory can be populated using ldapadd. In this example, a file containing this list of users is first created. Each user should use the following format: dn: dc=example,dc=com objectclass: dcObject objectclass: organization o: Example dc: Example dn: cn=Manager,dc=example,dc=com objectclass: organizationalRole cn: Manager To import this file, specify the file name. The following command will prompt for the password specified earlier and the output should look something like this: &prompt.root; ldapadd -Z -D "cn=Manager,dc=example,dc=com" -W -f import.ldif Enter LDAP Password: adding new entry "dc=example,dc=com" adding new entry "cn=Manager,dc=example,dc=com" Verify the data was added by issuing a search on the server using ldapsearch: &prompt.user; ldapsearch -Z # extended LDIF # # LDAPv3 # base <dc=example,dc=com> (default) with scope subtree # filter: (objectclass=*) # requesting: ALL # # example.com dn: dc=example,dc=com objectClass: dcObject objectClass: organization o: Example dc: Example # Manager, example.com dn: cn=Manager,dc=example,dc=com objectClass: organizationalRole cn: Manager # search result search: 3 result: 0 Success # numResponses: 3 # numEntries: 2 At this point, the server should be configured and functioning properly. Dynamic Host Configuration Protocol (<acronym>DHCP</acronym>) Dynamic Host Configuration Protocol DHCP Internet Systems Consortium (ISC) The Dynamic Host Configuration Protocol (DHCP) allows a system to connect to a network in order to be assigned the necessary addressing information for communication on that network. &os; includes the OpenBSD version of dhclient which is used by the client to obtain the addressing information. &os; does not install a DHCP server, but several servers are available in the &os; Ports Collection. The DHCP protocol is fully described in RFC 2131. Informational resources are also available at isc.org/downloads/dhcp/. This section describes how to use the built-in DHCP client. It then describes how to install and configure a DHCP server. In &os;, the &man.bpf.4; device is needed by both the DHCP server and DHCP client. This device is included in the GENERIC kernel that is installed with &os;. Users who prefer to create a custom kernel need to keep this device if DHCP is used. It should be noted that bpf also allows privileged users to run network packet sniffers on that system. Configuring a <acronym>DHCP</acronym> Client DHCP client support is included in the &os; installer, making it easy to configure a newly installed system to automatically receive its networking addressing information from an existing DHCP server. Refer to for examples of network configuration. UDP When dhclient is executed on the client machine, it begins broadcasting requests for configuration information. By default, these requests use UDP port 68. The server replies on UDP port 67, giving the client an IP address and other relevant network information such as a subnet mask, default gateway, and DNS server addresses. This information is in the form of a DHCP lease and is valid for a configurable time. This allows stale IP addresses for clients no longer connected to the network to automatically be reused. DHCP clients can obtain a great deal of information from the server. An exhaustive list may be found in &man.dhcp-options.5;. By default, when a &os; system boots, its DHCP client runs in the background, or asynchronously. Other startup scripts continue to run while the DHCP process completes, which speeds up system startup. Background DHCP works well when the DHCP server responds quickly to the client's requests. However, DHCP may take a long time to complete on some systems. If network services attempt to run before DHCP has assigned the network addressing information, they will fail. Using DHCP in synchronous mode prevents this problem as it pauses startup until the DHCP configuration has completed. This line in /etc/rc.conf is used to configure background or asynchronous mode: ifconfig_fxp0="DHCP" This line may already exist if the system was configured to use DHCP during installation. Replace the fxp0 shown in these examples with the name of the interface to be dynamically configured, as described in . To instead configure the system to use synchronous mode, and to pause during startup while DHCP completes, use SYNCDHCP: ifconfig_fxp0="SYNCDHCP" Additional client options are available. Search for dhclient in &man.rc.conf.5; for details. DHCP configuration files The DHCP client uses the following files: /etc/dhclient.conf The configuration file used by dhclient. Typically, this file contains only comments as the defaults are suitable for most clients. This configuration file is described in &man.dhclient.conf.5;. /sbin/dhclient More information about the command itself can be found in &man.dhclient.8;. /sbin/dhclient-script The &os;-specific DHCP client configuration script. It is described in &man.dhclient-script.8;, but should not need any user modification to function properly. /var/db/dhclient.leases.interface The DHCP client keeps a database of valid leases in this file, which is written as a log and is described in &man.dhclient.leases.5;. Installing and Configuring a <acronym>DHCP</acronym> Server This section demonstrates how to configure a &os; system to act as a DHCP server using the Internet Systems Consortium (ISC) implementation of the DHCP server. This implementation and its documentation can be installed using the net/isc-dhcp43-server package or port. DHCP server DHCP installation The installation of net/isc-dhcp43-server installs a sample configuration file. Copy /usr/local/etc/dhcpd.conf.example to /usr/local/etc/dhcpd.conf and make any edits to this new file. DHCP dhcpd.conf The configuration file is comprised of declarations for subnets and hosts which define the information that is provided to DHCP clients. For example, these lines configure the following: option domain-name "example.org"; option domain-name-servers ns1.example.org; option subnet-mask 255.255.255.0; default-lease-time 600; max-lease-time 72400; ddns-update-style none; subnet 10.254.239.0 netmask 255.255.255.224 { range 10.254.239.10 10.254.239.20; option routers rtr-239-0-1.example.org, rtr-239-0-2.example.org; } host fantasia { hardware ethernet 08:00:07:26:c0:a5; fixed-address fantasia.fugue.com; } This option specifies the default search domain that will be provided to clients. Refer to &man.resolv.conf.5; for more information. This option specifies a comma separated list of DNS servers that the client should use. They can be listed by their Fully Qualified Domain Names (FQDN), as seen in the example, or by their IP addresses. The subnet mask that will be provided to clients. The default lease expiry time in seconds. A client can be configured to override this value. The maximum allowed length of time, in seconds, for a lease. Should a client request a longer lease, a lease will still be issued, but it will only be valid for max-lease-time. The default of disables dynamic DNS updates. Changing this to configures the DHCP server to update a DNS server whenever it hands out a lease so that the DNS server knows which IP addresses are associated with which computers in the network. Do not change the default setting unless the DNS server has been configured to support dynamic DNS. This line creates a pool of available IP addresses which are reserved for allocation to DHCP clients. The range of addresses must be valid for the network or subnet specified in the previous line. Declares the default gateway that is valid for the network or subnet specified before the opening { bracket. Specifies the hardware MAC address of a client so that the DHCP server can recognize the client when it makes a request. Specifies that this host should always be given the same IP address. Using the hostname is correct, since the DHCP server will resolve the hostname before returning the lease information. This configuration file supports many more options. Refer to dhcpd.conf(5), installed with the server, for details and examples. Once the configuration of dhcpd.conf is complete, enable the DHCP server in /etc/rc.conf: dhcpd_enable="YES" dhcpd_ifaces="dc0" Replace the dc0 with the interface (or interfaces, separated by whitespace) that the DHCP server should listen on for DHCP client requests. Start the server by issuing the following command: &prompt.root; service isc-dhcpd start Any future changes to the configuration of the server will require the dhcpd service to be stopped and then started using &man.service.8;. The DHCP server uses the following files. Note that the manual pages are installed with the server software. DHCP configuration files /usr/local/sbin/dhcpd More information about the dhcpd server can be found in dhcpd(8). /usr/local/etc/dhcpd.conf The server configuration file needs to contain all the information that should be provided to clients, along with information regarding the operation of the server. This configuration file is described in dhcpd.conf(5). /var/db/dhcpd.leases The DHCP server keeps a database of leases it has issued in this file, which is written as a log. Refer to dhcpd.leases(5), which gives a slightly longer description. /usr/local/sbin/dhcrelay This daemon is used in advanced environments where one DHCP server forwards a request from a client to another DHCP server on a separate network. If this functionality is required, install the net/isc-dhcp43-relay package or port. The installation includes dhcrelay(8) which provides more detail. Domain Name System (<acronym>DNS</acronym>) DNS Domain Name System (DNS) is the protocol through which domain names are mapped to IP addresses, and vice versa. DNS is coordinated across the Internet through a somewhat complex system of authoritative root, Top Level Domain (TLD), and other smaller-scale name servers, which host and cache individual domain information. It is not necessary to run a name server to perform DNS lookups on a system. BIND In &os; 10, the Berkeley Internet Name Domain (BIND) has been removed from the base system and replaced with Unbound. Unbound as configured in the &os; Base is a local caching resolver. BIND is still available from The Ports Collection as dns/bind99 or dns/bind98. In &os; 9 and lower, BIND is included in &os; Base. The &os; version provides enhanced security features, a new file system layout, and automated &man.chroot.8; configuration. BIND is maintained by the Internet Systems Consortium. resolver reverse DNS root zone The following table describes some of the terms associated with DNS: <acronym>DNS</acronym> Terminology Term Definition Forward DNS Mapping of hostnames to IP addresses. Origin Refers to the domain covered in a particular zone file. named, BIND Common names for the BIND name server package within &os;. Resolver A system process through which a machine queries a name server for zone information. Reverse DNS Mapping of IP addresses to hostnames. Root zone The beginning of the Internet zone hierarchy. All zones fall under the root zone, similar to how all files in a file system fall under the root directory. Zone An individual domain, subdomain, or portion of the DNS administered by the same authority.
zones examples Examples of zones: . is how the root zone is usually referred to in documentation. org. is a Top Level Domain (TLD) under the root zone. example.org. is a zone under the org. TLD. 1.168.192.in-addr.arpa is a zone referencing all IP addresses which fall under the 192.168.1.* IP address space. As one can see, the more specific part of a hostname appears to its left. For example, example.org. is more specific than org., as org. is more specific than the root zone. The layout of each part of a hostname is much like a file system: the /dev directory falls within the root, and so on. Reasons to Run a Name Server Name servers generally come in two forms: authoritative name servers, and caching (also known as resolving) name servers. An authoritative name server is needed when: One wants to serve DNS information to the world, replying authoritatively to queries. A domain, such as example.org, is registered and IP addresses need to be assigned to hostnames under it. An IP address block requires reverse DNS entries (IP to hostname). A backup or second name server, called a slave, will reply to queries. A caching name server is needed when: A local DNS server may cache and respond more quickly than querying an outside name server. When one queries for www.FreeBSD.org, the resolver usually queries the uplink ISP's name server, and retrieves the reply. With a local, caching DNS server, the query only has to be made once to the outside world by the caching DNS server. Additional queries will not have to go outside the local network, since the information is cached locally. <acronym>DNS</acronym> Server Configuration in &os; 10.0 and Later In &os; 10.0, BIND has been replaced with Unbound. Unbound is a validating caching resolver only. If an authoritative server is needed, many are available from the Ports Collection. Unbound is provided in the &os; base system. By default, it will provide DNS resolution to the local machine only. While the base system package can be configured to provide resolution services beyond the local machine, it is recommended that such requirements be addressed by installing Unbound from the &os; Ports Collection. To enable Unbound, add the following to /etc/rc.conf: local_unbound_enable="YES" Any existing nameservers in /etc/resolv.conf will be configured as forwarders in the new Unbound configuration. If any of the listed nameservers do not support DNSSEC, local DNS resolution will fail. Be sure to test each nameserver and remove any that fail the test. The following command will show the trust tree or a failure for a nameserver running on 192.168.1.1: &prompt.user; drill -S FreeBSD.org @192.168.1.1 Once each nameserver is confirmed to support DNSSEC, start Unbound: &prompt.root; service local_unbound onestart This will take care of updating /etc/resolv.conf so that queries for DNSSEC secured domains will now work. For example, run the following to validate the FreeBSD.org DNSSEC trust tree: &prompt.user; drill -S FreeBSD.org ;; Number of trusted keys: 1 ;; Chasing: freebsd.org. A DNSSEC Trust tree: freebsd.org. (A) |---freebsd.org. (DNSKEY keytag: 36786 alg: 8 flags: 256) |---freebsd.org. (DNSKEY keytag: 32659 alg: 8 flags: 257) |---freebsd.org. (DS keytag: 32659 digest type: 2) |---org. (DNSKEY keytag: 49587 alg: 7 flags: 256) |---org. (DNSKEY keytag: 9795 alg: 7 flags: 257) |---org. (DNSKEY keytag: 21366 alg: 7 flags: 257) |---org. (DS keytag: 21366 digest type: 1) | |---. (DNSKEY keytag: 40926 alg: 8 flags: 256) | |---. (DNSKEY keytag: 19036 alg: 8 flags: 257) |---org. (DS keytag: 21366 digest type: 2) |---. (DNSKEY keytag: 40926 alg: 8 flags: 256) |---. (DNSKEY keytag: 19036 alg: 8 flags: 257) ;; Chase successful DNS Server Configuration in &os; 9.<replaceable>X</replaceable> In &os;, the BIND daemon is called named. File Description &man.named.8; The BIND daemon. &man.rndc.8; Name server control utility. /etc/namedb Directory where BIND zone information resides. /etc/namedb/named.conf Configuration file of the daemon. Depending on how a given zone is configured on the server, the files related to that zone can be found in the master, slave, or dynamic subdirectories of the /etc/namedb directory. These files contain the DNS information that will be given out by the name server in response to queries. Starting BIND BIND starting Since BIND is installed by default, configuring it is relatively simple. The default named configuration is that of a basic resolving name server, running in a &man.chroot.8; environment, and restricted to listening on the local IPv4 loopback address (127.0.0.1). To start the server one time with this configuration, use the following command: &prompt.root; service named onestart To ensure the named daemon is started at boot each time, put the following line into the /etc/rc.conf: named_enable="YES" There are many configuration options for /etc/namedb/named.conf that are beyond the scope of this document. Other startup options for named on &os; can be found in the named_* flags in /etc/defaults/rc.conf and in &man.rc.conf.5;. The section is also a good read. Configuration Files BIND configuration files Configuration files for named currently reside in /etc/namedb directory and will need modification before use unless all that is needed is a simple resolver. This is where most of the configuration will be performed. <filename>/etc/namedb/named.conf</filename> // $FreeBSD$ // // Refer to the named.conf(5) and named(8) man pages, and the documentation // in /usr/share/doc/bind9 for more details. // // If you are going to set up an authoritative server, make sure you // understand the hairy details of how DNS works. Even with // simple mistakes, you can break connectivity for affected parties, // or cause huge amounts of useless Internet traffic. options { // All file and path names are relative to the chroot directory, // if any, and should be fully qualified. directory "/etc/namedb/working"; pid-file "/var/run/named/pid"; dump-file "/var/dump/named_dump.db"; statistics-file "/var/stats/named.stats"; // If named is being used only as a local resolver, this is a safe default. // For named to be accessible to the network, comment this option, specify // the proper IP address, or delete this option. listen-on { 127.0.0.1; }; // If you have IPv6 enabled on this system, uncomment this option for // use as a local resolver. To give access to the network, specify // an IPv6 address, or the keyword "any". // listen-on-v6 { ::1; }; // These zones are already covered by the empty zones listed below. // If you remove the related empty zones below, comment these lines out. disable-empty-zone "255.255.255.255.IN-ADDR.ARPA"; disable-empty-zone "0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.IP6.ARPA"; disable-empty-zone "1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.IP6.ARPA"; // If you have a DNS server around at your upstream provider, enter // its IP address here, and enable the line below. This will make you // benefit from its cache, thus reduce overall DNS traffic in the Internet. /* forwarders { 127.0.0.1; }; */ // If the 'forwarders' clause is not empty the default is to 'forward first' // which will fall back to sending a query from your local server if the name // servers in 'forwarders' do not have the answer. Alternatively you can // force your name server to never initiate queries of its own by enabling the // following line: // forward only; // If you wish to have forwarding configured automatically based on // the entries in /etc/resolv.conf, uncomment the following line and // set named_auto_forward=yes in /etc/rc.conf. You can also enable // named_auto_forward_only (the effect of which is described above). // include "/etc/namedb/auto_forward.conf"; Just as the comment says, to benefit from an uplink's cache, forwarders can be enabled here. Under normal circumstances, a name server will recursively query the Internet looking at certain name servers until it finds the answer it is looking for. Having this enabled will have it query the uplink's name server (or name server provided) first, taking advantage of its cache. If the uplink name server in question is a heavily trafficked, fast name server, enabling this may be worthwhile. 127.0.0.1 will not work here. Change this IP address to a name server at the uplink. /* Modern versions of BIND use a random UDP port for each outgoing query by default in order to dramatically reduce the possibility of cache poisoning. All users are strongly encouraged to utilize this feature, and to configure their firewalls to accommodate it. AS A LAST RESORT in order to get around a restrictive firewall policy you can try enabling the option below. Use of this option will significantly reduce your ability to withstand cache poisoning attacks, and should be avoided if at all possible. Replace NNNNN in the example with a number between 49160 and 65530. */ // query-source address * port NNNNN; }; // If you enable a local name server, do not forget to enter 127.0.0.1 // first in your /etc/resolv.conf so this server will be queried. // Also, make sure to enable it in /etc/rc.conf. // The traditional root hints mechanism. Use this, OR the slave zones below. zone "." { type hint; file "/etc/namedb/named.root"; }; /* Slaving the following zones from the root name servers has some significant advantages: 1. Faster local resolution for your users 2. No spurious traffic will be sent from your network to the roots 3. Greater resilience to any potential root server failure/DDoS On the other hand, this method requires more monitoring than the hints file to be sure that an unexpected failure mode has not incapacitated your server. Name servers that are serving a lot of clients will benefit more from this approach than individual hosts. Use with caution. To use this mechanism, uncomment the entries below, and comment the hint zone above. As documented at http://dns.icann.org/services/axfr/ these zones: "." (the root), ARPA, IN-ADDR.ARPA, IP6.ARPA, and ROOT-SERVERS.NET are available for AXFR from these servers on IPv4 and IPv6: xfr.lax.dns.icann.org, xfr.cjr.dns.icann.org */ /* zone "." { type slave; file "/etc/namedb/slave/root.slave"; masters { 192.5.5.241; // F.ROOT-SERVERS.NET. }; notify no; }; zone "arpa" { type slave; file "/etc/namedb/slave/arpa.slave"; masters { 192.5.5.241; // F.ROOT-SERVERS.NET. }; notify no; }; */ /* Serving the following zones locally will prevent any queries for these zones leaving your network and going to the root name servers. This has two significant advantages: 1. Faster local resolution for your users 2. No spurious traffic will be sent from your network to the roots */ // RFCs 1912 and 5735 (and BCP 32 for localhost) zone "localhost" { type master; file "/etc/namedb/master/localhost-forward.db"; }; zone "127.in-addr.arpa" { type master; file "/etc/namedb/master/localhost-reverse.db"; }; zone "255.in-addr.arpa" { type master; file "/etc/namedb/master/empty.db"; }; // RFC 1912-style zone for IPv6 localhost address zone "0.ip6.arpa" { type master; file "/etc/namedb/master/localhost-reverse.db"; }; // "This" Network (RFCs 1912 and 5735) zone "0.in-addr.arpa" { type master; file "/etc/namedb/master/empty.db"; }; // Private Use Networks (RFCs 1918 and 5735) zone "10.in-addr.arpa" { type master; file "/etc/namedb/master/empty.db"; }; zone "16.172.in-addr.arpa" { type master; file "/etc/namedb/master/empty.db"; }; zone "17.172.in-addr.arpa" { type master; file "/etc/namedb/master/empty.db"; }; zone "18.172.in-addr.arpa" { type master; file "/etc/namedb/master/empty.db"; }; zone "19.172.in-addr.arpa" { type master; file "/etc/namedb/master/empty.db"; }; zone "20.172.in-addr.arpa" { type master; file "/etc/namedb/master/empty.db"; }; zone "21.172.in-addr.arpa" { type master; file "/etc/namedb/master/empty.db"; }; zone "22.172.in-addr.arpa" { type master; file "/etc/namedb/master/empty.db"; }; zone "23.172.in-addr.arpa" { type master; file "/etc/namedb/master/empty.db"; }; zone "24.172.in-addr.arpa" { type master; file "/etc/namedb/master/empty.db"; }; zone "25.172.in-addr.arpa" { type master; file "/etc/namedb/master/empty.db"; }; zone "26.172.in-addr.arpa" { type master; file "/etc/namedb/master/empty.db"; }; zone "27.172.in-addr.arpa" { type master; file "/etc/namedb/master/empty.db"; }; zone "28.172.in-addr.arpa" { type master; file "/etc/namedb/master/empty.db"; }; zone "29.172.in-addr.arpa" { type master; file "/etc/namedb/master/empty.db"; }; zone "30.172.in-addr.arpa" { type master; file "/etc/namedb/master/empty.db"; }; zone "31.172.in-addr.arpa" { type master; file "/etc/namedb/master/empty.db"; }; zone "168.192.in-addr.arpa" { type master; file "/etc/namedb/master/empty.db"; }; // Link-local/APIPA (RFCs 3927 and 5735) zone "254.169.in-addr.arpa" { type master; file "/etc/namedb/master/empty.db"; }; // IETF protocol assignments (RFCs 5735 and 5736) zone "0.0.192.in-addr.arpa" { type master; file "/etc/namedb/master/empty.db"; }; // TEST-NET-[1-3] for Documentation (RFCs 5735 and 5737) zone "2.0.192.in-addr.arpa" { type master; file "/etc/namedb/master/empty.db"; }; zone "100.51.198.in-addr.arpa" { type master; file "/etc/namedb/master/empty.db"; }; zone "113.0.203.in-addr.arpa" { type master; file "/etc/namedb/master/empty.db"; }; // IPv6 Range for Documentation (RFC 3849) zone "8.b.d.0.1.0.0.2.ip6.arpa" { type master; file "/etc/namedb/master/empty.db"; }; // Domain Names for Documentation and Testing (BCP 32) zone "test" { type master; file "/etc/namedb/master/empty.db"; }; zone "example" { type master; file "/etc/namedb/master/empty.db"; }; zone "invalid" { type master; file "/etc/namedb/master/empty.db"; }; zone "example.com" { type master; file "/etc/namedb/master/empty.db"; }; zone "example.net" { type master; file "/etc/namedb/master/empty.db"; }; zone "example.org" { type master; file "/etc/namedb/master/empty.db"; }; // Router Benchmark Testing (RFCs 2544 and 5735) zone "18.198.in-addr.arpa" { type master; file "/etc/namedb/master/empty.db"; }; zone "19.198.in-addr.arpa" { type master; file "/etc/namedb/master/empty.db"; }; // IANA Reserved - Old Class E Space (RFC 5735) zone "240.in-addr.arpa" { type master; file "/etc/namedb/master/empty.db"; }; zone "241.in-addr.arpa" { type master; file "/etc/namedb/master/empty.db"; }; zone "242.in-addr.arpa" { type master; file "/etc/namedb/master/empty.db"; }; zone "243.in-addr.arpa" { type master; file "/etc/namedb/master/empty.db"; }; zone "244.in-addr.arpa" { type master; file "/etc/namedb/master/empty.db"; }; zone "245.in-addr.arpa" { type master; file "/etc/namedb/master/empty.db"; }; zone "246.in-addr.arpa" { type master; file "/etc/namedb/master/empty.db"; }; zone "247.in-addr.arpa" { type master; file "/etc/namedb/master/empty.db"; }; zone "248.in-addr.arpa" { type master; file "/etc/namedb/master/empty.db"; }; zone "249.in-addr.arpa" { type master; file "/etc/namedb/master/empty.db"; }; zone "250.in-addr.arpa" { type master; file "/etc/namedb/master/empty.db"; }; zone "251.in-addr.arpa" { type master; file "/etc/namedb/master/empty.db"; }; zone "252.in-addr.arpa" { type master; file "/etc/namedb/master/empty.db"; }; zone "253.in-addr.arpa" { type master; file "/etc/namedb/master/empty.db"; }; zone "254.in-addr.arpa" { type master; file "/etc/namedb/master/empty.db"; }; // IPv6 Unassigned Addresses (RFC 4291) zone "1.ip6.arpa" { type master; file "/etc/namedb/master/empty.db"; }; zone "3.ip6.arpa" { type master; file "/etc/namedb/master/empty.db"; }; zone "4.ip6.arpa" { type master; file "/etc/namedb/master/empty.db"; }; zone "5.ip6.arpa" { type master; file "/etc/namedb/master/empty.db"; }; zone "6.ip6.arpa" { type master; file "/etc/namedb/master/empty.db"; }; zone "7.ip6.arpa" { type master; file "/etc/namedb/master/empty.db"; }; zone "8.ip6.arpa" { type master; file "/etc/namedb/master/empty.db"; }; zone "9.ip6.arpa" { type master; file "/etc/namedb/master/empty.db"; }; zone "a.ip6.arpa" { type master; file "/etc/namedb/master/empty.db"; }; zone "b.ip6.arpa" { type master; file "/etc/namedb/master/empty.db"; }; zone "c.ip6.arpa" { type master; file "/etc/namedb/master/empty.db"; }; zone "d.ip6.arpa" { type master; file "/etc/namedb/master/empty.db"; }; zone "e.ip6.arpa" { type master; file "/etc/namedb/master/empty.db"; }; zone "0.f.ip6.arpa" { type master; file "/etc/namedb/master/empty.db"; }; zone "1.f.ip6.arpa" { type master; file "/etc/namedb/master/empty.db"; }; zone "2.f.ip6.arpa" { type master; file "/etc/namedb/master/empty.db"; }; zone "3.f.ip6.arpa" { type master; file "/etc/namedb/master/empty.db"; }; zone "4.f.ip6.arpa" { type master; file "/etc/namedb/master/empty.db"; }; zone "5.f.ip6.arpa" { type master; file "/etc/namedb/master/empty.db"; }; zone "6.f.ip6.arpa" { type master; file "/etc/namedb/master/empty.db"; }; zone "7.f.ip6.arpa" { type master; file "/etc/namedb/master/empty.db"; }; zone "8.f.ip6.arpa" { type master; file "/etc/namedb/master/empty.db"; }; zone "9.f.ip6.arpa" { type master; file "/etc/namedb/master/empty.db"; }; zone "a.f.ip6.arpa" { type master; file "/etc/namedb/master/empty.db"; }; zone "b.f.ip6.arpa" { type master; file "/etc/namedb/master/empty.db"; }; zone "0.e.f.ip6.arpa" { type master; file "/etc/namedb/master/empty.db"; }; zone "1.e.f.ip6.arpa" { type master; file "/etc/namedb/master/empty.db"; }; zone "2.e.f.ip6.arpa" { type master; file "/etc/namedb/master/empty.db"; }; zone "3.e.f.ip6.arpa" { type master; file "/etc/namedb/master/empty.db"; }; zone "4.e.f.ip6.arpa" { type master; file "/etc/namedb/master/empty.db"; }; zone "5.e.f.ip6.arpa" { type master; file "/etc/namedb/master/empty.db"; }; zone "6.e.f.ip6.arpa" { type master; file "/etc/namedb/master/empty.db"; }; zone "7.e.f.ip6.arpa" { type master; file "/etc/namedb/master/empty.db"; }; // IPv6 ULA (RFC 4193) zone "c.f.ip6.arpa" { type master; file "/etc/namedb/master/empty.db"; }; zone "d.f.ip6.arpa" { type master; file "/etc/namedb/master/empty.db"; }; // IPv6 Link Local (RFC 4291) zone "8.e.f.ip6.arpa" { type master; file "/etc/namedb/master/empty.db"; }; zone "9.e.f.ip6.arpa" { type master; file "/etc/namedb/master/empty.db"; }; zone "a.e.f.ip6.arpa" { type master; file "/etc/namedb/master/empty.db"; }; zone "b.e.f.ip6.arpa" { type master; file "/etc/namedb/master/empty.db"; }; // IPv6 Deprecated Site-Local Addresses (RFC 3879) zone "c.e.f.ip6.arpa" { type master; file "/etc/namedb/master/empty.db"; }; zone "d.e.f.ip6.arpa" { type master; file "/etc/namedb/master/empty.db"; }; zone "e.e.f.ip6.arpa" { type master; file "/etc/namedb/master/empty.db"; }; zone "f.e.f.ip6.arpa" { type master; file "/etc/namedb/master/empty.db"; }; // IP6.INT is Deprecated (RFC 4159) zone "ip6.int" { type master; file "/etc/namedb/master/empty.db"; }; // NB: Do not use the IP addresses below, they are faked, and only // serve demonstration/documentation purposes! // // Example slave zone config entries. It can be convenient to become // a slave at least for the zone your own domain is in. Ask // your network administrator for the IP address of the responsible // master name server. // // Do not forget to include the reverse lookup zone! // This is named after the first bytes of the IP address, in reverse // order, with ".IN-ADDR.ARPA" appended, or ".IP6.ARPA" for IPv6. // // Before starting to set up a master zone, make sure you fully // understand how DNS and BIND work. There are sometimes // non-obvious pitfalls. Setting up a slave zone is usually simpler. // // NB: Do not blindly enable the examples below. :-) Use actual names // and addresses instead. /* An example dynamic zone key "exampleorgkey" { algorithm hmac-md5; secret "sf87HJqjkqh8ac87a02lla=="; }; zone "example.org" { type master; allow-update { key "exampleorgkey"; }; file "/etc/namedb/dynamic/example.org"; }; */ /* Example of a slave reverse zone zone "1.168.192.in-addr.arpa" { type slave; file "/etc/namedb/slave/1.168.192.in-addr.arpa"; masters { 192.168.1.1; }; }; */ In named.conf, these are examples of slave entries for a forward and reverse zone. For each new zone served, a new zone entry must be added to named.conf. For example, the simplest zone entry for example.org can look like: zone "example.org" { type master; file "master/example.org"; }; The zone is a master, as indicated by the statement, holding its zone information in /etc/namedb/master/example.org indicated by the statement. zone "example.org" { type slave; file "slave/example.org"; }; In the slave case, the zone information is transferred from the master name server for the particular zone, and saved in the file specified. If and when the master server dies or is unreachable, the slave name server will have the transferred zone information and will be able to serve it. Zone Files BIND zone files An example master zone file for example.org (existing within /etc/namedb/master/example.org) is as follows: $TTL 3600 ; 1 hour default TTL example.org. IN SOA ns1.example.org. admin.example.org. ( 2006051501 ; Serial 10800 ; Refresh 3600 ; Retry 604800 ; Expire 300 ; Negative Response TTL ) ; DNS Servers IN NS ns1.example.org. IN NS ns2.example.org. ; MX Records IN MX 10 mx.example.org. IN MX 20 mail.example.org. IN A 192.168.1.1 ; Machine Names localhost IN A 127.0.0.1 ns1 IN A 192.168.1.2 ns2 IN A 192.168.1.3 mx IN A 192.168.1.4 mail IN A 192.168.1.5 ; Aliases www IN CNAME example.org. Note that every hostname ending in a . is an exact hostname, whereas everything without a trailing . is relative to the origin. For example, ns1 is translated into ns1.example.org. The format of a zone file follows: recordname IN recordtype value DNS records The most commonly used DNS records: SOA start of zone authority NS an authoritative name server A a host address CNAME the canonical name for an alias MX mail exchanger PTR a domain name pointer (used in reverse DNS) example.org. IN SOA ns1.example.org. admin.example.org. ( 2006051501 ; Serial 10800 ; Refresh after 3 hours 3600 ; Retry after 1 hour 604800 ; Expire after 1 week 300 ) ; Negative Response TTL example.org. the domain name, also the origin for this zone file. ns1.example.org. the primary/authoritative name server for this zone. admin.example.org. the responsible person for this zone, email address with @ replaced. (admin@example.org becomes admin.example.org) 2006051501 the serial number of the file. This must be incremented each time the zone file is modified. Nowadays, many admins prefer a yyyymmddrr format for the serial number. 2006051501 would mean last modified 05/15/2006, the latter 01 being the first time the zone file has been modified this day. The serial number is important as it alerts slave name servers for a zone when it is updated. IN NS ns1.example.org. This is an NS entry. Every name server that is going to reply authoritatively for the zone must have one of these entries. localhost IN A 127.0.0.1 ns1 IN A 192.168.1.2 ns2 IN A 192.168.1.3 mx IN A 192.168.1.4 mail IN A 192.168.1.5 The A record indicates machine names. As seen above, ns1.example.org would resolve to 192.168.1.2. IN A 192.168.1.1 This line assigns IP address 192.168.1.1 to the current origin, in this case example.org. www IN CNAME @ The canonical name record is usually used for giving aliases to a machine. In the example, www is aliased to the master machine whose name happens to be the same as the domain name example.org (192.168.1.1). CNAMEs can never be used together with another kind of record for the same hostname. MX record IN MX 10 mail.example.org. The MX record indicates which mail servers are responsible for handling incoming mail for the zone. mail.example.org is the hostname of a mail server, and 10 is the priority of that mail server. One can have several mail servers, with priorities of 10, 20 and so on. A mail server attempting to deliver to example.org would first try the highest priority MX (the record with the lowest priority number), then the second highest, etc, until the mail can be properly delivered. For in-addr.arpa zone files (reverse DNS), the same format is used, except with PTR entries instead of A or CNAME. $TTL 3600 1.168.192.in-addr.arpa. IN SOA ns1.example.org. admin.example.org. ( 2006051501 ; Serial 10800 ; Refresh 3600 ; Retry 604800 ; Expire 300 ) ; Negative Response TTL IN NS ns1.example.org. IN NS ns2.example.org. 1 IN PTR example.org. 2 IN PTR ns1.example.org. 3 IN PTR ns2.example.org. 4 IN PTR mx.example.org. 5 IN PTR mail.example.org. This file gives the proper IP address to hostname mappings for the above fictitious domain. It is worth noting that all names on the right side of a PTR record need to be fully qualified (i.e., end in a .). Caching Name Server BIND caching name server A caching name server is a name server whose primary role is to resolve recursive queries. It simply asks queries of its own, and remembers the answers for later use. <acronym role="Domain Name Security Extensions">DNSSEC</acronym> BIND DNS security extensions Domain Name System Security Extensions, or DNSSEC for short, is a suite of specifications to protect resolving name servers from forged DNS data, such as spoofed DNS records. By using digital signatures, a resolver can verify the integrity of the record. Note that DNSSEC only provides integrity via digitally signing the Resource Records (RRs). It provides neither confidentiality nor protection against false end-user assumptions. This means that it cannot protect against people going to example.net instead of example.com. The only thing DNSSEC does is authenticate that the data has not been compromised in transit. The security of DNS is an important step in securing the Internet in general. For more in-depth details of how DNSSEC works, the relevant RFCs are a good place to start. See the list in . The following sections will demonstrate how to enable DNSSEC for an authoritative DNS server and a recursive (or caching) DNS server running BIND 9. While all versions of BIND 9 support DNSSEC, it is necessary to have at least version 9.6.2 in order to be able to use the signed root zone when validating DNS queries. This is because earlier versions lack the required algorithms to enable validation using the root zone key. It is strongly recommended to use the latest version of BIND 9.7 or later to take advantage of automatic key updating for the root key, as well as other features to automatically keep zones signed and signatures up to date. Where configurations differ between 9.6.2 and 9.7 and later, differences will be pointed out. Recursive <acronym>DNS</acronym> Server Configuration Enabling DNSSEC validation of queries performed by a recursive DNS server requires a few changes to named.conf. Before making these changes the root zone key, or trust anchor, must be acquired. Currently the root zone key is not available in a file format BIND understands, so it has to be manually converted into the proper format. The key itself can be obtained by querying the root zone for it using dig. By running &prompt.user; dig +multi +noall +answer DNSKEY . > root.dnskey the key will end up in root.dnskey. The contents should look something like this: . 93910 IN DNSKEY 257 3 8 ( AwEAAagAIKlVZrpC6Ia7gEzahOR+9W29euxhJhVVLOyQ bSEW0O8gcCjFFVQUTf6v58fLjwBd0YI0EzrAcQqBGCzh /RStIoO8g0NfnfL2MTJRkxoXbfDaUeVPQuYEhg37NZWA JQ9VnMVDxP/VHL496M/QZxkjf5/Efucp2gaDX6RS6CXp oY68LsvPVjR0ZSwzz1apAzvN9dlzEheX7ICJBBtuA6G3 LQpzW5hOA2hzCTMjJPJ8LbqF6dsV6DoBQzgul0sGIcGO Yl7OyQdXfZ57relSQageu+ipAdTTJ25AsRTAoub8ONGc LmqrAmRLKBP1dfwhYB4N7knNnulqQxA+Uk1ihz0= ) ; key id = 19036 . 93910 IN DNSKEY 256 3 8 ( AwEAAcaGQEA+OJmOzfzVfoYN249JId7gx+OZMbxy69Hf UyuGBbRN0+HuTOpBxxBCkNOL+EJB9qJxt+0FEY6ZUVjE g58sRr4ZQ6Iu6b1xTBKgc193zUARk4mmQ/PPGxn7Cn5V EGJ/1h6dNaiXuRHwR+7oWh7DnzkIJChcTqlFrXDW3tjt ) ; key id = 34525 Do not be alarmed if the obtained keys differ from this example. They might have changed since these instructions were last updated. This output actually contains two keys. The first key in the listing, with the value 257 after the DNSKEY record type, is the one needed. This value indicates that this is a Secure Entry Point (SEP), commonly known as a Key Signing Key (KSK). The second key, with value 256, is a subordinate key, commonly called a Zone Signing Key (ZSK). More on the different key types later in . Now the key must be verified and formatted so that BIND can use it. To verify the key, generate a DS RR set. Create a file containing these RRs with &prompt.user; dnssec-dsfromkey -f root.dnskey . > root.ds These records use SHA-1 and SHA-256 respectively, and should look similar to the following example, where the longer is using SHA-256. . IN DS 19036 8 1 B256BD09DC8DD59F0E0F0D8541B8328DD986DF6E . IN DS 19036 8 2 49AAC11D7B6F6446702E54A1607371607A1A41855200FD2CE1CDDE32F24E8FB5 The SHA-256 RR can now be compared to the digest in https://data.iana.org/root-anchors/root-anchors.xml. To be absolutely sure that the key has not been tampered with the data in the XML file should be verified using a proper PGP signature. Next, the key must be formatted properly. This differs a little between BIND versions 9.6.2 and 9.7 and later. In version 9.7 support was added to automatically track changes to the key and update it as necessary. This is done using managed-keys as seen in the example below. When using the older version, the key is added using a trusted-keys statement and updates must be done manually. For BIND 9.6.2 the format should look like: trusted-keys { "." 257 3 8 "AwEAAagAIKlVZrpC6Ia7gEzahOR+9W29euxhJhVVLOyQbSEW0O8gcCjF FVQUTf6v58fLjwBd0YI0EzrAcQqBGCzh/RStIoO8g0NfnfL2MTJRkxoX bfDaUeVPQuYEhg37NZWAJQ9VnMVDxP/VHL496M/QZxkjf5/Efucp2gaD X6RS6CXpoY68LsvPVjR0ZSwzz1apAzvN9dlzEheX7ICJBBtuA6G3LQpz W5hOA2hzCTMjJPJ8LbqF6dsV6DoBQzgul0sGIcGOYl7OyQdXfZ57relS Qageu+ipAdTTJ25AsRTAoub8ONGcLmqrAmRLKBP1dfwhYB4N7knNnulq QxA+Uk1ihz0="; }; For 9.7 the format will instead be: managed-keys { "." initial-key 257 3 8 "AwEAAagAIKlVZrpC6Ia7gEzahOR+9W29euxhJhVVLOyQbSEW0O8gcCjF FVQUTf6v58fLjwBd0YI0EzrAcQqBGCzh/RStIoO8g0NfnfL2MTJRkxoX bfDaUeVPQuYEhg37NZWAJQ9VnMVDxP/VHL496M/QZxkjf5/Efucp2gaD X6RS6CXpoY68LsvPVjR0ZSwzz1apAzvN9dlzEheX7ICJBBtuA6G3LQpz W5hOA2hzCTMjJPJ8LbqF6dsV6DoBQzgul0sGIcGOYl7OyQdXfZ57relS Qageu+ipAdTTJ25AsRTAoub8ONGcLmqrAmRLKBP1dfwhYB4N7knNnulq QxA+Uk1ihz0="; }; The root key can now be added to named.conf either directly or by including a file containing the key. After these steps, configure BIND to do DNSSEC validation on queries by editing named.conf and adding the following to the options directive: dnssec-enable yes; dnssec-validation yes; To verify that it is actually working use dig to make a query for a signed zone using the resolver just configured. A successful reply will contain the AD flag to indicate the data was authenticated. Running a query such as &prompt.user; dig @resolver +dnssec se ds should return the DS RR for the .se zone. In the flags: section the AD flag should be set, as seen in: ... ;; flags: qr rd ra ad; QUERY: 1, ANSWER: 3, AUTHORITY: 0, ADDITIONAL: 1 ... The resolver is now capable of authenticating DNS queries. Authoritative <acronym>DNS</acronym> Server Configuration In order to get an authoritative name server to serve a DNSSEC signed zone a little more work is required. A zone is signed using cryptographic keys which must be generated. It is possible to use only one key for this. The preferred method however is to have a strong well-protected Key Signing Key (KSK) that is not rotated very often and a Zone Signing Key (ZSK) that is rotated more frequently. Information on recommended operational practices can be found in RFC 4641: DNSSEC Operational Practices. Practices regarding the root zone can be found in DNSSEC Practice Statement for the Root Zone KSK operator and DNSSEC Practice Statement for the Root Zone ZSK operator. The KSK is used to build a chain of authority to the data in need of validation and as such is also called a Secure Entry Point (SEP) key. A message digest of this key, called a Delegation Signer (DS) record, must be published in the parent zone to establish the trust chain. How this is accomplished depends on the parent zone owner. The ZSK is used to sign the zone, and only needs to be published there. To enable DNSSEC for the example.com zone depicted in previous examples, the first step is to use dnssec-keygen to generate the KSK and ZSK key pair. This key pair can utilize different cryptographic algorithms. It is recommended to use RSA/SHA256 for the keys and 2048 bits key length should be enough. To generate the KSK for example.com, run &prompt.user; dnssec-keygen -f KSK -a RSASHA256 -b 2048 -n ZONE example.com and to generate the ZSK, run &prompt.user; dnssec-keygen -a RSASHA256 -b 2048 -n ZONE example.com dnssec-keygen outputs two files, the public and the private keys in files named similar to Kexample.com.+005+nnnnn.key (public) and Kexample.com.+005+nnnnn.private (private). The nnnnn part of the file name is a five digit key ID. Keep track of which key ID belongs to which key. This is especially important when having more than one key in a zone. It is also possible to rename the keys. For each KSK file do: &prompt.user; mv Kexample.com.+005+nnnnn.key Kexample.com.+005+nnnnn.KSK.key &prompt.user; mv Kexample.com.+005+nnnnn.private Kexample.com.+005+nnnnn.KSK.private For the ZSK files, substitute KSK for ZSK as necessary. The files can now be included in the zone file, using the $include statement. It should look something like this: $include Kexample.com.+005+nnnnn.KSK.key ; KSK $include Kexample.com.+005+nnnnn.ZSK.key ; ZSK Finally, sign the zone and tell BIND to use the signed zone file. To sign a zone dnssec-signzone is used. The command to sign the zone example.com, located in example.com.db would look similar to - &prompt.user; dnssec-signzone -o - example.com -k Kexample.com.+005+nnnnn.KSK example.com.db - Kexample.com.+005+nnnnn.ZSK.key + &prompt.user; dnssec-signzone -o example.com -k Kexample.com.+005+nnnnn.KSK example.com.db Kexample.com.+005+nnnnn.ZSK.key The key supplied to the argument is the KSK and the other key file is the ZSK that should be used in the signing. It is possible to supply more than one KSK and ZSK, which will result in the zone being signed with all supplied keys. This can be needed to supply zone data signed using more than one algorithm. The output of dnssec-signzone is a zone file with all RRs signed. This output will end up in a file with the extension .signed, such as example.com.db.signed. The DS records will also be written to a separate file dsset-example.com. To use this signed zone just modify the zone directive in named.conf to use example.com.db.signed. By default, the signatures are only valid 30 days, meaning that the zone needs to be resigned in about 15 days to be sure that resolvers are not caching records with stale signatures. It is possible to make a script and a cron job to do this. See relevant manuals for details. Be sure to keep private keys confidential, as with all cryptographic keys. When changing a key it is best to include the new key into the zone, while still signing with the old one, and then move over to using the new key to sign. After these steps are done the old key can be removed from the zone. Failure to do this might render the DNS data unavailable for a time, until the new key has propagated through the DNS hierarchy. For more information on key rollovers and other DNSSEC operational issues, see RFC 4641: DNSSEC Operational practices. Automation Using <acronym>BIND</acronym> 9.7 or Later Beginning with BIND version 9.7 a new feature called Smart Signing was introduced. This feature aims to make the key management and signing process simpler by automating parts of the task. By putting the keys into a directory called a key repository, and using the new option auto-dnssec, it is possible to create a dynamic zone which will be resigned as needed. To update this zone use nsupdate with the new option . rndc has also grown the ability to sign zones with keys in the key repository, using the option . To tell BIND to use this automatic signing and zone updating for example.com, add the following to named.conf: zone example.com { type master; key-directory "/etc/named/keys"; update-policy local; auto-dnssec maintain; file "/etc/named/dynamic/example.com.zone"; }; After making these changes, generate keys for the zone as explained in , put those keys in the key repository given as the argument to the key-directory in the zone configuration and the zone will be signed automatically. Updates to a zone configured this way must be done using nsupdate, which will take care of re-signing the zone with the new data added. For further details, see and the BIND documentation. Security Although BIND is the most common implementation of DNS, there is always the issue of security. Possible and exploitable security holes are sometimes found. While &os; automatically drops named into a &man.chroot.8; environment; there are several other security mechanisms in place which could help to lure off possible DNS service attacks. It is always good idea to read CERT's security advisories and to subscribe to the &a.security-notifications; to stay up to date with the current Internet and &os; security issues. If a problem arises, keeping sources up to date and having a fresh build of named may help. Further Reading BIND/named manual pages: &man.rndc.8; &man.named.8; &man.named.conf.5; &man.nsupdate.1; &man.dnssec-signzone.8; &man.dnssec-keygen.8; Official ISC BIND Page Official ISC BIND Forum O'Reilly DNS and BIND 5th Edition Root DNSSEC DNSSEC Trust Anchor Publication for the Root Zone RFC1034 - Domain Names - Concepts and Facilities RFC1035 - Domain Names - Implementation and Specification RFC4033 - DNS Security Introduction and Requirements RFC4034 - Resource Records for the DNS Security Extensions RFC4035 - Protocol Modifications for the DNS Security Extensions RFC4641 - DNSSEC Operational Practices RFC 5011 - Automated Updates of DNS Security (DNSSEC Trust Anchors Apache HTTP Server Murray Stokely Contributed by web servers setting up Apache The open source Apache HTTP Server is the most widely used web server. &os; does not install this web server by default, but it can be installed from the www/apache24 package or port. This section summarizes how to configure and start version 2.x of the Apache HTTP Server on &os;. For more detailed information about Apache 2.X and its configuration directives, refer to httpd.apache.org. Configuring and Starting Apache Apache configuration file In &os;, the main Apache HTTP Server configuration file is installed as /usr/local/etc/apache2x/httpd.conf, where x represents the version number. This ASCII text file begins comment lines with a #. The most frequently modified directives are: ServerRoot "/usr/local" Specifies the default directory hierarchy for the Apache installation. Binaries are stored in the bin and sbin subdirectories of the server root and configuration files are stored in the etc/apache2x subdirectory. ServerAdmin you@example.com Change this to the email address to receive problems with the server. This address also appears on some server-generated pages, such as error documents. ServerName www.example.com:80 Allows an administrator to set a hostname which is sent back to clients for the server. For example, www can be used instead of the actual hostname. If the system does not have a registered DNS name, enter its IP address instead. If the server will listen on an alternate report, change 80 to the alternate port number. DocumentRoot "/usr/local/www/apache2x/data" The directory where documents will be served from. By default, all requests are taken from this directory, but symbolic links and aliases may be used to point to other locations. It is always a good idea to make a backup copy of the default Apache configuration file before making changes. When the configuration of Apache is complete, save the file and verify the configuration using apachectl. Running apachectl configtest should return Syntax OK. Apache starting or stopping To launch Apache at system startup, add the following line to /etc/rc.conf: apache24_enable="YES" If Apache should be started with non-default options, the following line may be added to /etc/rc.conf to specify the needed flags: apache24_flags="" If apachectl does not report configuration errors, start httpd now: &prompt.root; service apache24 start The httpd service can be tested by entering http://localhost in a web browser, replacing localhost with the fully-qualified domain name of the machine running httpd. The default web page that is displayed is /usr/local/www/apache24/data/index.html. The Apache configuration can be tested for errors after making subsequent configuration changes while httpd is running using the following command: &prompt.root; service apache24 configtest It is important to note that configtest is not an &man.rc.8; standard, and should not be expected to work for all startup scripts. Virtual Hosting Virtual hosting allows multiple websites to run on one Apache server. The virtual hosts can be IP-based or name-based. IP-based virtual hosting uses a different IP address for each website. Name-based virtual hosting uses the clients HTTP/1.1 headers to figure out the hostname, which allows the websites to share the same IP address. To setup Apache to use name-based virtual hosting, add a VirtualHost block for each website. For example, for the webserver named www.domain.tld with a virtual domain of www.someotherdomain.tld, add the following entries to httpd.conf: <VirtualHost *> ServerName www.domain.tld DocumentRoot /www/domain.tld </VirtualHost> <VirtualHost *> ServerName www.someotherdomain.tld DocumentRoot /www/someotherdomain.tld </VirtualHost> For each virtual host, replace the values for ServerName and DocumentRoot with the values to be used. For more information about setting up virtual hosts, consult the official Apache documentation at: http://httpd.apache.org/docs/vhosts/. Apache Modules Apache modules Apache uses modules to augment the functionality provided by the basic server. Refer to http://httpd.apache.org/docs/current/mod/ for a complete listing of and the configuration details for the available modules. In &os;, some modules can be compiled with the www/apache24 port. Type make config within /usr/ports/www/apache24 to see which modules are available and which are enabled by default. If the module is not compiled with the port, the &os; Ports Collection provides an easy way to install many modules. This section describes three of the most commonly used modules. <filename>mod_ssl</filename> web servers secure SSL cryptography The mod_ssl module uses the OpenSSL library to provide strong cryptography via the Secure Sockets Layer (SSLv3) and Transport Layer Security (TLSv1) protocols. This module provides everything necessary to request a signed certificate from a trusted certificate signing authority to run a secure web server on &os;. In &os;, mod_ssl module is enabled by default in both the package and the port. The available configuration directives are explained at http://httpd.apache.org/docs/current/mod/mod_ssl.html. <filename>mod_perl</filename> mod_perl Perl The mod_perl module makes it possible to write Apache modules in Perl. In addition, the persistent interpreter embedded in the server avoids the overhead of starting an external interpreter and the penalty of Perl start-up time. The mod_perl can be installed using the www/mod_perl2 package or port. Documentation for using this module can be found at http://perl.apache.org/docs/2.0/index.html. <filename>mod_php</filename> Tom Rhodes Written by mod_php PHP PHP: Hypertext Preprocessor (PHP) is a general-purpose scripting language that is especially suited for web development. Capable of being embedded into HTML, its syntax draws upon C, &java;, and Perl with the intention of allowing web developers to write dynamically generated webpages quickly. To gain support for PHP5 for the Apache web server, install the www/mod_php56 package or port. This will install and configure the modules required to support dynamic PHP applications. The installation will automatically add this line to /usr/local/etc/apache24/httpd.conf: LoadModule php5_module libexec/apache24/libphp5.so Then, perform a graceful restart to load the PHP module: &prompt.root; apachectl graceful The PHP support provided by www/mod_php56 is limited. Additional support can be installed using the lang/php56-extensions port which provides a menu driven interface to the available PHP extensions. Alternatively, individual extensions can be installed using the appropriate port. For instance, to add PHP support for the MySQL database server, install databases/php56-mysql. After installing an extension, the Apache server must be reloaded to pick up the new configuration changes: &prompt.root; apachectl graceful Dynamic Websites web servers dynamic In addition to mod_perl and mod_php, other languages are available for creating dynamic web content. These include Django and Ruby on Rails. Django Python Django Django is a BSD-licensed framework designed to allow developers to write high performance, elegant web applications quickly. It provides an object-relational mapper so that data types are developed as Python objects. A rich dynamic database-access API is provided for those objects without the developer ever having to write SQL. It also provides an extensible template system so that the logic of the application is separated from the HTML presentation. Django depends on mod_python, and an SQL database engine. In &os;, the www/py-django port automatically installs mod_python and supports the PostgreSQL, MySQL, or SQLite databases, with the default being SQLite. To change the database engine, type make config within /usr/ports/www/py-django, then install the port. Once Django is installed, the application will need a project directory along with the Apache configuration in order to use the embedded Python interpreter. This interpreter is used to call the application for specific URLs on the site. To configure Apache to pass requests for certain URLs to the web application, add the following to httpd.conf, specifying the full path to the project directory: <Location "/"> SetHandler python-program PythonPath "['/dir/to/the/django/packages/'] + sys.path" PythonHandler django.core.handlers.modpython SetEnv DJANGO_SETTINGS_MODULE mysite.settings PythonAutoReload On PythonDebug On </Location> Refer to https://docs.djangoproject.com for more information on how to use Django. Ruby on Rails Ruby on Rails Ruby on Rails is another open source web framework that provides a full development stack. It is optimized to make web developers more productive and capable of writing powerful applications quickly. On &os;, it can be installed using the www/rubygem-rails package or port. Refer to http://guides.rubyonrails.org for more information on how to use Ruby on Rails. File Transfer Protocol (<acronym>FTP</acronym>) FTP servers The File Transfer Protocol (FTP) provides users with a simple way to transfer files to and from an FTP server. &os; includes FTP server software, ftpd, in the base system. &os; provides several configuration files for controlling access to the FTP server. This section summarizes these files. Refer to &man.ftpd.8; for more details about the built-in FTP server. Configuration The most important configuration step is deciding which accounts will be allowed access to the FTP server. A &os; system has a number of system accounts which should not be allowed FTP access. The list of users disallowed any FTP access can be found in /etc/ftpusers. By default, it includes system accounts. Additional users that should not be allowed access to FTP can be added. In some cases it may be desirable to restrict the access of some users without preventing them completely from using FTP. This can be accomplished be creating /etc/ftpchroot as described in &man.ftpchroot.5;. This file lists users and groups subject to FTP access restrictions. FTP anonymous To enable anonymous FTP access to the server, create a user named ftp on the &os; system. Users will then be able to log on to the FTP server with a username of ftp or anonymous. When prompted for the password, any input will be accepted, but by convention, an email address should be used as the password. The FTP server will call &man.chroot.2; when an anonymous user logs in, to restrict access to only the home directory of the ftp user. There are two text files that can be created to specify welcome messages to be displayed to FTP clients. The contents of /etc/ftpwelcome will be displayed to users before they reach the login prompt. After a successful login, the contents of /etc/ftpmotd will be displayed. Note that the path to this file is relative to the login environment, so the contents of ~ftp/etc/ftpmotd would be displayed for anonymous users. Once the FTP server has been configured, set the appropriate variable in /etc/rc.conf to start the service during boot: ftpd_enable="YES" To start the service now: &prompt.root; service ftpd start Test the connection to the FTP server by typing: &prompt.user; ftp localhost syslog log files FTP The ftpd daemon uses &man.syslog.3; to log messages. By default, the system log daemon will write messages related to FTP in /var/log/xferlog. The location of the FTP log can be modified by changing the following line in /etc/syslog.conf: ftp.info /var/log/xferlog FTP anonymous Be aware of the potential problems involved with running an anonymous FTP server. In particular, think twice about allowing anonymous users to upload files. It may turn out that the FTP site becomes a forum for the trade of unlicensed commercial software or worse. If anonymous FTP uploads are required, then verify the permissions so that these files cannot be read by other anonymous users until they have been reviewed by an administrator. File and Print Services for µsoft.windows; Clients (Samba) Samba server Microsoft Windows file server Windows clients print server Windows clients Samba is a popular open source software package that provides file and print services using the SMB/CIFS protocol. This protocol is built into µsoft.windows; systems. It can be added to non-µsoft.windows; systems by installing the Samba client libraries. The protocol allows clients to access shared data and printers. These shares can be mapped as a local disk drive and shared printers can be used as if they were local printers. On &os;, the Samba client libraries can be installed using the net/samba-smbclient port or package. The client provides the ability for a &os; system to access SMB/CIFS shares in a µsoft.windows; network. A &os; system can also be configured to act as a Samba server by installing the net/samba43 port or package. This allows the administrator to create SMB/CIFS shares on the &os; system which can be accessed by clients running µsoft.windows; or the Samba client libraries. Server Configuration Samba is configured in /usr/local/etc/smb4.conf. This file must be created before Samba can be used. A simple smb4.conf to share directories and printers with &windows; clients in a workgroup is shown here. For more complex setups involving LDAP or Active Directory, it is easier to use &man.samba-tool.8; to create the initial smb4.conf. [global] workgroup = WORKGROUP server string = Samba Server Version %v netbios name = ExampleMachine wins support = Yes security = user passdb backend = tdbsam # Example: share /usr/src accessible only to 'developer' user [src] path = /usr/src valid users = developer writable = yes browsable = yes read only = no guest ok = no public = no create mask = 0666 directory mask = 0755 Global Settings Settings that describe the network are added in /usr/local/etc/smb4.conf: workgroup The name of the workgroup to be served. netbios name The NetBIOS name by which a Samba server is known. By default, it is the same as the first component of the host's DNS name. server string The string that will be displayed in the output of net view and some other networking tools that seek to display descriptive text about the server. wins support Whether Samba will act as a WINS server. Do not enable support for WINS on more than one server on the network. Security Settings The most important settings in /usr/local/etc/smb4.conf are the security model and the backend password format. These directives control the options: security The most common settings are security = share and security = user. If the clients use usernames that are the same as their usernames on the &os; machine, user level security should be used. This is the default security policy and it requires clients to first log on before they can access shared resources. In share level security, clients do not need to log onto the server with a valid username and password before attempting to connect to a shared resource. This was the default security model for older versions of Samba. passdb backend NIS+ LDAP SQL database Samba has several different backend authentication models. Clients may be authenticated with LDAP, NIS+, an SQL database, or a modified password file. The recommended authentication method, tdbsam, is ideal for simple networks and is covered here. For larger or more complex networks, ldapsam is recommended. smbpasswd was the former default and is now obsolete. <application>Samba</application> Users &os; user accounts must be mapped to the SambaSAMAccount database for &windows; clients to access the share. Map existing &os; user accounts using &man.pdbedit.8;: &prompt.root; pdbedit -a username This section has only mentioned the most commonly used settings. Refer to the Official Samba HOWTO for additional information about the available configuration options. Starting <application>Samba</application> To enable Samba at boot time, add the following line to /etc/rc.conf: samba_enable="YES" To enable Samba4, use: samba_server_enable="YES" To start Samba now: &prompt.root; service samba start Starting SAMBA: removing stale tdbs : Starting nmbd. Starting smbd. Samba consists of three separate daemons. Both the nmbd and smbd daemons are started by samba_enable. If winbind name resolution is also required, set: winbindd_enable="YES" Samba can be stopped at any time by typing: &prompt.root; service samba stop Samba is a complex software suite with functionality that allows broad integration with µsoft.windows; networks. For more information about functionality beyond the basic configuration described here, refer to http://www.samba.org. Clock Synchronization with NTP NTP ntpd Over time, a computer's clock is prone to drift. This is problematic as many network services require the computers on a network to share the same accurate time. Accurate time is also needed to ensure that file timestamps stay consistent. The Network Time Protocol (NTP) is one way to provide clock accuracy in a network. &os; includes &man.ntpd.8; which can be configured to query other NTP servers in order to synchronize the clock on that machine or to provide time services to other computers in the network. The servers which are queried can be local to the network or provided by an ISP. In addition, an online list of publicly accessible NTP servers is available. When choosing a public NTP server, select one that is geographically close and review its usage policy. Choosing several NTP servers is recommended in case one of the servers becomes unreachable or its clock proves unreliable. As ntpd receives responses, it favors reliable servers over the less reliable ones. This section describes how to configure ntpd on &os;. Further documentation can be found in /usr/share/doc/ntp/ in HTML format. <acronym>NTP</acronym> Configuration NTP ntp.conf On &os;, the built-in ntpd can be used to synchronize a system's clock. To enable ntpd at boot time, add ntpd_enable="YES" to /etc/rc.conf. Additional variables can be specified in /etc/rc.conf. Refer to &man.rc.conf.5; and &man.ntpd.8; for details. This application reads /etc/ntp.conf to determine which NTP servers to query. Here is a simple example of an /etc/ntp.conf: Sample <filename>/etc/ntp.conf</filename> server ntplocal.example.com prefer server timeserver.example.org server ntp2a.example.net driftfile /var/db/ntp.drift The format of this file is described in &man.ntp.conf.5;. The server option specifies which servers to query, with one server listed on each line. If a server entry includes prefer, that server is preferred over other servers. A response from a preferred server will be discarded if it differs significantly from other servers' responses; otherwise it will be used. The prefer argument should only be used for NTP servers that are known to be highly accurate, such as those with special time monitoring hardware. The driftfile entry specifies which file is used to store the system clock's frequency offset. ntpd uses this to automatically compensate for the clock's natural drift, allowing it to maintain a reasonably correct setting even if it is cut off from all external time sources for a period of time. This file also stores information about previous responses from NTP servers. Since this file contains internal information for NTP, it should not be modified. By default, an NTP server is accessible to any network host. The restrict option in /etc/ntp.conf can be used to control which systems can access the server. For example, to deny all machines from accessing the NTP server, add the following line to /etc/ntp.conf: restrict default ignore This will also prevent access from other NTP servers. If there is a need to synchronize with an external NTP server, allow only that specific server. Refer to &man.ntp.conf.5; for more information. To allow machines within the network to synchronize their clocks with the server, but ensure they are not allowed to configure the server or be used as peers to synchronize against, instead use: restrict 192.168.1.0 mask 255.255.255.0 nomodify notrap where 192.168.1.0 is the local network address and 255.255.255.0 is the network's subnet mask. Multiple restrict entries are supported. For more details, refer to the Access Control Support subsection of &man.ntp.conf.5;. Once ntpd_enable="YES" has been added to /etc/rc.conf, ntpd can be started now without rebooting the system by typing: &prompt.root; service ntpd start Using <acronym>NTP</acronym> with a <acronym>PPP</acronym> Connection ntpd does not need a permanent connection to the Internet to function properly. However, if a PPP connection is configured to dial out on demand, NTP traffic should be prevented from triggering a dial out or keeping the connection alive. This can be configured with filter directives in /etc/ppp/ppp.conf. For example: set filter dial 0 deny udp src eq 123 # Prevent NTP traffic from initiating dial out set filter dial 1 permit 0 0 set filter alive 0 deny udp src eq 123 # Prevent incoming NTP traffic from keeping the connection open set filter alive 1 deny udp dst eq 123 # Prevent outgoing NTP traffic from keeping the connection open set filter alive 2 permit 0/0 0/0 For more details, refer to the PACKET FILTERING section in &man.ppp.8; and the examples in /usr/share/examples/ppp/. Some Internet access providers block low-numbered ports, preventing NTP from functioning since replies never reach the machine. <acronym>iSCSI</acronym> Initiator and Target Configuration iSCSI is a way to share storage over a network. Unlike NFS, which works at the file system level, iSCSI works at the block device level. In iSCSI terminology, the system that shares the storage is known as the target. The storage can be a physical disk, or an area representing multiple disks or a portion of a physical disk. For example, if the disk(s) are formatted with ZFS, a zvol can be created to use as the iSCSI storage. The clients which access the iSCSI storage are called initiators. To initiators, the storage available through iSCSI appears as a raw, unformatted disk known as a LUN. Device nodes for the disk appear in /dev/ and the device must be separately formatted and mounted. Beginning with 10.0-RELEASE, &os; provides a native, kernel-based iSCSI target and initiator. This section describes how to configure a &os; system as a target or an initiator. Configuring an <acronym>iSCSI</acronym> Target The native iSCSI target is supported starting with &os; 10.0-RELEASE. To use iSCSI in older versions of &os;, install a userspace target from the Ports Collection, such as net/istgt. This chapter only describes the native target. To configure an iSCSI target, create the /etc/ctl.conf configuration file, add a line to /etc/rc.conf to make sure the &man.ctld.8; daemon is automatically started at boot, and then start the daemon. The following is an example of a simple /etc/ctl.conf configuration file. Refer to &man.ctl.conf.5; for a more complete description of this file's available options. portal-group pg0 { discovery-auth-group no-authentication listen 0.0.0.0 listen [::] } target iqn.2012-06.com.example:target0 { auth-group no-authentication portal-group pg0 lun 0 { path /data/target0-0 size 4G } } The first entry defines the pg0 portal group. Portal groups define which network addresses the &man.ctld.8; daemon will listen on. The discovery-auth-group no-authentication entry indicates that any initiator is allowed to perform iSCSI target discovery without authentication. Lines three and four configure &man.ctld.8; to listen on all IPv4 (listen 0.0.0.0) and IPv6 (listen [::]) addresses on the default port of 3260. It is not necessary to define a portal group as there is a built-in portal group called default. In this case, the difference between default and pg0 is that with default, target discovery is always denied, while with pg0, it is always allowed. The second entry defines a single target. Target has two possible meanings: a machine serving iSCSI or a named group of LUNs. This example uses the latter meaning, where iqn.2012-06.com.example:target0 is the target name. This target name is suitable for testing purposes. For actual use, change com.example to the real domain name, reversed. The 2012-06 represents the year and month of acquiring control of that domain name, and target0 can be any value. Any number of targets can be defined in this configuration file. The auth-group no-authentication line allows all initiators to connect to the specified target and portal-group pg0 makes the target reachable through the pg0 portal group. The next section defines the LUN. To the initiator, each LUN will be visible as a separate disk device. Multiple LUNs can be defined for each target. Each LUN is identified by a number, where LUN 0 is mandatory. The path /data/target0-0 line defines the full path to a file or zvol backing the LUN. That path must exist before starting &man.ctld.8;. The second line is optional and specifies the size of the LUN. Next, to make sure the &man.ctld.8; daemon is started at boot, add this line to /etc/rc.conf: ctld_enable="YES" To start &man.ctld.8; now, run this command: &prompt.root; service ctld start As the &man.ctld.8; daemon is started, it reads /etc/ctl.conf. If this file is edited after the daemon starts, use this command so that the changes take effect immediately: &prompt.root; service ctld reload Authentication The previous example is inherently insecure as it uses no authentication, granting anyone full access to all targets. To require a username and password to access targets, modify the configuration as follows: auth-group ag0 { chap username1 secretsecret chap username2 anothersecret } portal-group pg0 { discovery-auth-group no-authentication listen 0.0.0.0 listen [::] } target iqn.2012-06.com.example:target0 { auth-group ag0 portal-group pg0 lun 0 { path /data/target0-0 size 4G } } The auth-group section defines username and password pairs. An initiator trying to connect to iqn.2012-06.com.example:target0 must first specify a defined username and secret. However, target discovery is still permitted without authentication. To require target discovery authentication, set discovery-auth-group to a defined auth-group name instead of no-authentication. It is common to define a single exported target for every initiator. As a shorthand for the syntax above, the username and password can be specified directly in the target entry: target iqn.2012-06.com.example:target0 { portal-group pg0 chap username1 secretsecret lun 0 { path /data/target0-0 size 4G } } Configuring an <acronym>iSCSI</acronym> Initiator The iSCSI initiator described in this section is supported starting with &os; 10.0-RELEASE. To use the iSCSI initiator available in older versions, refer to &man.iscontrol.8;. The iSCSI initiator requires that the &man.iscsid.8; daemon is running. This daemon does not use a configuration file. To start it automatically at boot, add this line to /etc/rc.conf: iscsid_enable="YES" To start &man.iscsid.8; now, run this command: &prompt.root; service iscsid start Connecting to a target can be done with or without an /etc/iscsi.conf configuration file. This section demonstrates both types of connections. Connecting to a Target Without a Configuration File To connect an initiator to a single target, specify the IP address of the portal and the name of the target: &prompt.root; iscsictl -A -p 10.10.10.10 -t iqn.2012-06.com.example:target0 To verify if the connection succeeded, run iscsictl without any arguments. The output should look similar to this: Target name Target portal State iqn.2012-06.com.example:target0 10.10.10.10 Connected: da0 In this example, the iSCSI session was successfully established, with /dev/da0 representing the attached LUN. If the iqn.2012-06.com.example:target0 target exports more than one LUN, multiple device nodes will be shown in that section of the output: Connected: da0 da1 da2. Any errors will be reported in the output, as well as the system logs. For example, this message usually means that the &man.iscsid.8; daemon is not running: Target name Target portal State iqn.2012-06.com.example:target0 10.10.10.10 Waiting for iscsid(8) The following message suggests a networking problem, such as a wrong IP address or port: Target name Target portal State iqn.2012-06.com.example:target0 10.10.10.11 Connection refused This message means that the specified target name is wrong: Target name Target portal State iqn.2012-06.com.example:target0 10.10.10.10 Not found This message means that the target requires authentication: Target name Target portal State iqn.2012-06.com.example:target0 10.10.10.10 Authentication failed To specify a CHAP username and secret, use this syntax: &prompt.root; iscsictl -A -p 10.10.10.10 -t iqn.2012-06.com.example:target0 -u user -s secretsecret Connecting to a Target with a Configuration File To connect using a configuration file, create /etc/iscsi.conf with contents like this: t0 { TargetAddress = 10.10.10.10 TargetName = iqn.2012-06.com.example:target0 AuthMethod = CHAP chapIName = user chapSecret = secretsecret } The t0 specifies a nickname for the configuration file section. It will be used by the initiator to specify which configuration to use. The other lines specify the parameters to use during connection. The TargetAddress and TargetName are mandatory, whereas the other options are optional. In this example, the CHAP username and secret are shown. To connect to the defined target, specify the nickname: &prompt.root; iscsictl -An t0 Alternately, to connect to all targets defined in the configuration file, use: &prompt.root; iscsictl -Aa To make the initiator automatically connect to all targets in /etc/iscsi.conf, add the following to /etc/rc.conf: iscsictl_enable="YES" iscsictl_flags="-Aa" Index: head/en_US.ISO8859-1/books/handbook/virtualization/chapter.xml =================================================================== --- head/en_US.ISO8859-1/books/handbook/virtualization/chapter.xml (revision 50804) +++ head/en_US.ISO8859-1/books/handbook/virtualization/chapter.xml (revision 50805) @@ -1,1716 +1,1715 @@ Virtualization Murray Stokely Contributed by Allan Jude bhyve section by Benedict Reuschling Xen section by Synopsis Virtualization software allows multiple operating systems to run simultaneously on the same computer. Such software systems for PCs often involve a host operating system which runs the virtualization software and supports any number of guest operating systems. After reading this chapter, you will know: The difference between a host operating system and a guest operating system. How to install &os; on an &intel;-based &apple; &mac; computer. How to install &os; on µsoft.windows; with Virtual PC. How to install &os; as a guest in bhyve. How to tune a &os; system for best performance under virtualization. Before reading this chapter, you should: Understand the basics of &unix; and &os;. Know how to install &os;. Know how to set up a network connection. Know how to install additional third-party software. &os; as a Guest on <application>Parallels</application> for &macos; X Parallels Desktop for &mac; is a commercial software product available for &intel; based &apple; &mac; computers running &macos; 10.4.6 or higher. &os; is a fully supported guest operating system. Once Parallels has been installed on &macos; X, the user must configure a virtual machine and then install the desired guest operating system. Installing &os; on Parallels/&macos; X The first step in installing &os; on Parallels is to create a new virtual machine for installing &os;. Select &os; as the Guest OS Type when prompted: Choose a reasonable amount of disk and memory depending on the plans for this virtual &os; instance. 4GB of disk space and 512MB of RAM work well for most uses of &os; under Parallels: Select the type of networking and a network interface: Save and finish the configuration: After the &os; virtual machine has been created, &os; can be installed on it. This is best done with an official &os; CD/DVD or with an ISO image downloaded from an official FTP site. Copy the appropriate ISO image to the local &mac; filesystem or insert a CD/DVD in the &mac;'s CD-ROM drive. Click on the disc icon in the bottom right corner of the &os; Parallels window. This will bring up a window that can be used to associate the CD-ROM drive in the virtual machine with the ISO file on disk or with the real CD-ROM drive. Once this association with the CD-ROM source has been made, reboot the &os; virtual machine by clicking the reboot icon. Parallels will reboot with a special BIOS that first checks if there is a CD-ROM. In this case it will find the &os; installation media and begin a normal &os; installation. Perform the installation, but do not attempt to configure &xorg; at this time. When the installation is finished, reboot into the newly installed &os; virtual machine. Configuring &os; on <application>Parallels</application> After &os; has been successfully installed on &macos; X with Parallels, there are a number of configuration steps that can be taken to optimize the system for virtualized operation. Set Boot Loader Variables The most important step is to reduce the tunable to reduce the CPU utilization of &os; under the Parallels environment. This is accomplished by adding the following line to /boot/loader.conf: kern.hz=100 Without this setting, an idle &os; Parallels guest will use roughly 15% of the CPU of a single processor &imac;. After this change the usage will be closer to 5%. Create a New Kernel Configuration File All of the SCSI, FireWire, and USB device drivers can be removed from a custom kernel configuration file. Parallels provides a virtual network adapter used by the &man.ed.4; driver, so all network devices except for &man.ed.4; and &man.miibus.4; can be removed from the kernel. Configure Networking The most basic networking setup uses DHCP to connect the virtual machine to the same local area network as the host &mac;. This can be accomplished by adding ifconfig_ed0="DHCP" to /etc/rc.conf. More advanced networking setups are described in . &os; as a Guest on <application>Virtual PC</application> for &windows; Virtual PC for &windows; is a µsoft; software product available for free download. See this website for the system requirements. Once Virtual PC has been installed on µsoft.windows;, the user can configure a virtual machine and then install the desired guest operating system. Installing &os; on <application>Virtual PC</application> The first step in installing &os; on Virtual PC is to create a new virtual machine for installing &os;. Select Create a virtual machine when prompted: Select Other as the Operating system when prompted: Then, choose a reasonable amount of disk and memory depending on the plans for this virtual &os; instance. 4GB of disk space and 512MB of RAM work well for most uses of &os; under Virtual PC: Save and finish the configuration: Select the &os; virtual machine and click Settings, then set the type of networking and a network interface: After the &os; virtual machine has been created, &os; can be installed on it. This is best done with an official &os; CD/DVD or with an ISO image downloaded from an official FTP site. Copy the appropriate ISO image to the local &windows; filesystem or insert a CD/DVD in the CD drive, then double click on the &os; virtual machine to boot. Then, click CD and choose Capture ISO Image... on the Virtual PC window. This will bring up a window where the CD-ROM drive in the virtual machine can be associated with an ISO file on disk or with the real CD-ROM drive. Once this association with the CD-ROM source has been made, reboot the &os; virtual machine by clicking Action and Reset. Virtual PC will reboot with a special BIOS that first checks for a CD-ROM. In this case it will find the &os; installation media and begin a normal &os; installation. Continue with the installation, but do not attempt to configure &xorg; at this time. When the installation is finished, remember to eject the CD/DVD or release the ISO image. Finally, reboot into the newly installed &os; virtual machine. Configuring &os; on <application>Virtual PC</application> After &os; has been successfully installed on µsoft.windows; with Virtual PC, there are a number of configuration steps that can be taken to optimize the system for virtualized operation. Set Boot Loader Variables The most important step is to reduce the tunable to reduce the CPU utilization of &os; under the Virtual PC environment. This is accomplished by adding the following line to /boot/loader.conf: kern.hz=100 Without this setting, an idle &os; Virtual PC guest OS will use roughly 40% of the CPU of a single processor computer. After this change, the usage will be closer to 3%. Create a New Kernel Configuration File All of the SCSI, FireWire, and USB device drivers can be removed from a custom kernel configuration file. Virtual PC provides a virtual network adapter used by the &man.de.4; driver, so all network devices except for &man.de.4; and &man.miibus.4; can be removed from the kernel. Configure Networking The most basic networking setup uses DHCP to connect the virtual machine to the same local area network as the µsoft.windows; host. This can be accomplished by adding ifconfig_de0="DHCP" to /etc/rc.conf. More advanced networking setups are described in . &os; as a Guest on <application>VMware Fusion</application> for &macos; VMware Fusion for &mac; is a commercial software product available for &intel; based &apple; &mac; computers running &macos; 10.4.9 or higher. &os; is a fully supported guest operating system. Once VMware Fusion has been installed on &macos; X, the user can configure a virtual machine and then install the desired guest operating system. Installing &os; on <application>VMware Fusion</application> The first step is to start VMware Fusion which will load the Virtual Machine Library. Click New to create the virtual machine: This will load the New Virtual Machine Assistant. Click Continue to proceed: Select Other as the Operating System and either &os; or &os; 64-bit, as the Version when prompted: Choose the name of the virtual machine and the directory where it should be saved: Choose the size of the Virtual Hard Disk for the virtual machine: Choose the method to install the virtual machine, either from an ISO image or from a CD/DVD: Click Finish and the virtual machine will boot: Install &os; as usual: Once the install is complete, the settings of the virtual machine can be modified, such as memory usage: The System Hardware settings of the virtual machine cannot be modified while the virtual machine is running. The number of CPUs the virtual machine will have access to: The status of the CD-ROM device. Normally the CD/DVD/ISO is disconnected from the virtual machine when it is no longer needed. The last thing to change is how the virtual machine will connect to the network. To allow connections to the virtual machine from other machines besides the host, choose Connect directly to the physical network (Bridged). Otherwise, Share the host's internet connection (NAT) is preferred so that the virtual machine can have access to the Internet, but the network cannot access the virtual machine. After modifying the settings, boot the newly installed &os; virtual machine. Configuring &os; on <application>VMware Fusion</application> After &os; has been successfully installed on &macos; X with VMware Fusion, there are a number of configuration steps that can be taken to optimize the system for virtualized operation. Set Boot Loader Variables The most important step is to reduce the tunable to reduce the CPU utilization of &os; under the VMware Fusion environment. This is accomplished by adding the following line to /boot/loader.conf: kern.hz=100 Without this setting, an idle &os; VMware Fusion guest will use roughly 15% of the CPU of a single processor &imac;. After this change, the usage will be closer to 5%. Create a New Kernel Configuration File All of the FireWire, and USB device drivers can be removed from a custom kernel configuration file. VMware Fusion provides a virtual network adapter used by the &man.em.4; driver, so all network devices except for &man.em.4; can be removed from the kernel. Configure Networking The most basic networking setup uses DHCP to connect the virtual machine to the same local area network as the host &mac;. This can be accomplished by adding ifconfig_em0="DHCP" to /etc/rc.conf. More advanced networking setups are described in . &os; as a Guest on &virtualbox; &os; works well as a guest in &virtualbox;. The virtualization software is available for most common operating systems, including &os; itself. The &virtualbox; guest additions provide support for: Clipboard sharing. Mouse pointer integration. Host time synchronization. Window scaling. Seamless mode. These commands are run in the &os; guest. First, install the emulators/virtualbox-ose-additions package or port in the &os; guest. This will install the port: &prompt.root; cd /usr/ports/emulators/virtualbox-ose-additions && make install clean Add these lines to /etc/rc.conf: vboxguest_enable="YES" vboxservice_enable="YES" If &man.ntpd.8; or &man.ntpdate.8; is used, disable host time synchronization: vboxservice_flags="--disable-timesync" Xorg will automatically recognize the vboxvideo driver. It can also be manually entered in /etc/X11/xorg.conf: Section "Device" Identifier "Card0" Driver "vboxvideo" VendorName "InnoTek Systemberatung GmbH" BoardName "VirtualBox Graphics Adapter" EndSection To use the vboxmouse driver, adjust the mouse section in /etc/X11/xorg.conf: Section "InputDevice" Identifier "Mouse0" Driver "vboxmouse" EndSection HAL users should create the following /usr/local/etc/hal/fdi/policy/90-vboxguest.fdi or copy it from /usr/local/share/hal/fdi/policy/10osvendor/90-vboxguest.fdi: <?xml version="1.0" encoding="utf-8"?> <!-- # Sun VirtualBox # Hal driver description for the vboxmouse driver # $Id: chapter.xml,v 1.33 2012-03-17 04:53:52 eadler Exp $ Copyright (C) 2008-2009 Sun Microsystems, Inc. This file is part of VirtualBox Open Source Edition (OSE, as available from http://www.virtualbox.org. This file is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License (GPL) as published by the Free Software Foundation, in version 2 as it comes in the "COPYING" file of the VirtualBox OSE distribution. VirtualBox OSE is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY of any kind. Please contact Sun Microsystems, Inc., 4150 Network Circle, Santa Clara, CA 95054 USA or visit http://www.sun.com if you need additional information or have any questions. --> <deviceinfo version="0.2"> <device> <match key="info.subsystem" string="pci"> <match key="info.product" string="VirtualBox guest Service"> <append key="info.capabilities" type="strlist">input</append> <append key="info.capabilities" type="strlist">input.mouse</append> <merge key="input.x11_driver" type="string">vboxmouse</merge> <merge key="input.device" type="string">/dev/vboxguest</merge> </match> </match> </device> </deviceinfo> &os; as a Host with <application>VirtualBox</application> &virtualbox; is an actively developed, complete virtualization package, that is available for most operating systems including &windows;, &macos;, &linux; and &os;. It is equally capable of running &windows; or &unix;-like guests. It is released as open source software, but with closed-source components available in a separate extension pack. These components include support for USB 2.0 devices. More information may be found on the Downloads page of the &virtualbox; wiki. Currently, these extensions are not available for &os;. Installing &virtualbox; &virtualbox; is available as a &os; package or port in emulators/virtualbox-ose. The port can be installed using these commands: &prompt.root; cd /usr/ports/emulators/virtualbox-ose &prompt.root; make install clean One useful option in the port's configuration menu is the GuestAdditions suite of programs. These provide a number of useful features in guest operating systems, like mouse pointer integration (allowing the mouse to be shared between host and guest without the need to press a special keyboard shortcut to switch) and faster video rendering, especially in &windows; guests. The guest additions are available in the Devices menu, after the installation of the guest is finished. A few configuration changes are needed before &virtualbox; is started for the first time. The port installs a kernel module in /boot/modules which must be loaded into the running kernel: &prompt.root; kldload vboxdrv To ensure the module is always loaded after a reboot, add this line to /boot/loader.conf: vboxdrv_load="YES" To use the kernel modules that allow bridged or host-only networking, add this line to /etc/rc.conf and reboot the computer: vboxnet_enable="YES" The vboxusers group is created during installation of &virtualbox;. All users that need access to &virtualbox; will have to be added as members of this group. pw can be used to add new members: &prompt.root; pw groupmod vboxusers -m yourusername The default permissions for /dev/vboxnetctl are restrictive and need to be changed for bridged networking: &prompt.root; chown root:vboxusers /dev/vboxnetctl &prompt.root; chmod 0660 /dev/vboxnetctl To make this permissions change permanent, add these lines to /etc/devfs.conf: own vboxnetctl root:vboxusers perm vboxnetctl 0660 To launch &virtualbox;, type from a &xorg; session: &prompt.user; VirtualBox For more information on configuring and using &virtualbox;, refer to the official website. For &os;-specific information and troubleshooting instructions, refer to the relevant page in the &os; wiki. &virtualbox; USB Support &virtualbox; can be configured to pass USB devices through to the guest operating system. The host controller of the OSE version is limited to emulating USB 1.1 devices until the extension pack supporting USB 2.0 and 3.0 devices becomes available on &os;. For &virtualbox; to be aware of USB devices attached to the machine, the user needs to be a member of the operator group. - &prompt.root; pw groupmod operator -m - yourusername + &prompt.root; pw groupmod operator -m yourusername Restart the login session and &virtualbox; for these changes to take effect, and create USB filters as necessary. &virtualbox; Host <acronym>DVD</acronym>/<acronym>CD</acronym> Access Access to the host DVD/CD drives from guests is achieved through the sharing of the physical drives. Within &virtualbox;, this is set up from the Storage window in the Settings of the virtual machine. If needed, create an empty IDE CD/DVD device first. Then choose the Host Drive from the popup menu for the virtual CD/DVD drive selection. A checkbox labeled Passthrough will appear. This allows the virtual machine to use the hardware directly. For example, audio CDs or the burner will only function if this option is selected. HAL needs to run for &virtualbox; DVD/CD functions to work, so enable it in /etc/rc.conf and start it if it is not already running: hald_enable="YES" &prompt.root; service hald start In order for users to be able to use &virtualbox; DVD/CD functions, they need access to /dev/xpt0, /dev/cdN, and /dev/passN. This is usually achieved by making the user a member of operator. Permissions to these devices have to be corrected by adding these lines to /etc/devfs.conf: perm cd* 0660 perm xpt0 0660 perm pass* 0660 &prompt.root; service devfs restart &os; as a Host with <application>bhyve</application> The bhyve BSD-licensed hypervisor became part of the base system with &os; 10.0-RELEASE. This hypervisor supports a number of guests, including &os;, OpenBSD, and many &linux; distributions. By default, bhyve provides access to serial console and does not emulate a graphical console. Virtualization offload features of newer CPUs are used to avoid the legacy methods of translating instructions and manually managing memory mappings. The bhyve design requires a processor that supports &intel; Extended Page Tables (EPT) or &amd; Rapid Virtualization Indexing (RVI) or Nested Page Tables (NPT). Hosting &linux; guests or &os; guests with more than one vCPU requires VMX unrestricted mode support (UG). Most newer processors, specifically the &intel; &core; i3/i5/i7 and &intel; &xeon; E3/E5/E7, support these features. UG support was introduced with Intel's Westmere micro-architecture. For a complete list of &intel; processors that support EPT, refer to . RVI is found on the third generation and later of the &amd.opteron; (Barcelona) processors. The easiest way to tell if a processor supports bhyve is to run dmesg or look in /var/run/dmesg.boot for the POPCNT processor feature flag on the Features2 line for &amd; processors or EPT and UG on the VT-x line for &intel; processors. Preparing the Host The first step to creating a virtual machine in bhyve is configuring the host system. First, load the bhyve kernel module: &prompt.root; kldload vmm Then, create a tap interface for the network device in the virtual machine to attach to. In order for the network device to participate in the network, also create a bridge interface containing the tap interface and the physical interface as members. In this example, the physical interface is igb0: &prompt.root; ifconfig tap0 create &prompt.root; sysctl net.link.tap.up_on_open=1 net.link.tap.up_on_open: 0 -> 1 &prompt.root; ifconfig bridge0 create &prompt.root; ifconfig bridge0 addm igb0 addm tap0 &prompt.root; ifconfig bridge0 up Creating a FreeBSD Guest Create a file to use as the virtual disk for the guest machine. Specify the size and name of the virtual disk: &prompt.root; truncate -s 16G guest.img Download an installation image of &os; to install: &prompt.root; fetch ftp://ftp.freebsd.org/pub/FreeBSD/releases/ISO-IMAGES/10.3/FreeBSD-10.3-RELEASE-amd64-bootonly.iso FreeBSD-10.3-RELEASE-amd64-bootonly.iso 100% of 230 MB 570 kBps 06m17s &os; comes with an example script for running a virtual machine in bhyve. The script will start the virtual machine and run it in a loop, so it will automatically restart if it crashes. The script takes a number of options to control the configuration of the machine: controls the number of virtual CPUs, limits the amount of memory available to the guest, defines which tap device to use, indicates which disk image to use, tells bhyve to boot from the CD image instead of the disk, and defines which CD image to use. The last parameter is the name of the virtual machine, used to track the running machines. This example starts the virtual machine in installation mode: &prompt.root; sh /usr/share/examples/bhyve/vmrun.sh -c 1 -m 1024M -t tap0 -d guest.img -i -I FreeBSD-10.3-RELEASE-amd64-bootonly.iso guestname The virtual machine will boot and start the installer. After installing a system in the virtual machine, when the system asks about dropping in to a shell at the end of the installation, choose Yes. A small change needs to be made to make the system start with a serial console. Edit /etc/ttys and replace the existing ttyu0 line with: ttyu0 "/usr/libexec/getty 3wire" xterm on secure Beginning with &os; 9.3-RELEASE and 10.1-RELEASE the console is configured automatically. Reboot the virtual machine. While rebooting the virtual machine causes bhyve to exit, the vmrun.sh script runs bhyve in a loop and will automatically restart it. When this happens, choose the reboot option from the boot loader menu in order to escape the loop. Now the guest can be started from the virtual disk: &prompt.root; sh /usr/share/examples/bhyve/vmrun.sh -c 4 -m 1024M -t tap0 -d guest.img guestname Creating a &linux; Guest In order to boot operating systems other than &os;, the sysutils/grub2-bhyve port must be first installed. Next, create a file to use as the virtual disk for the guest machine: &prompt.root; truncate -s 16G linux.img Starting a virtual machine with bhyve is a two step process. First a kernel must be loaded, then the guest can be started. The &linux; kernel is loaded with sysutils/grub2-bhyve. Create a device.map that grub will use to map the virtual devices to the files on the host system: (hd0) ./linux.img (cd0) ./somelinux.iso Use sysutils/grub2-bhyve to load the &linux; kernel from the ISO image: &prompt.root; grub-bhyve -m device.map -r cd0 -M 1024M linuxguest This will start grub. If the installation CD contains a grub.cfg, a menu will be displayed. If not, the vmlinuz and initrd files must be located and loaded manually: grub> ls (hd0) (cd0) (cd0,msdos1) (host) grub> ls (cd0)/isolinux boot.cat boot.msg grub.conf initrd.img isolinux.bin isolinux.cfg memtest splash.jpg TRANS.TBL vesamenu.c32 vmlinuz grub> linux (cd0)/isolinux/vmlinuz grub> initrd (cd0)/isolinux/initrd.img grub> boot Now that the &linux; kernel is loaded, the guest can be started: &prompt.root; bhyve -A -H -P -s 0:0,hostbridge -s 1:0,lpc -s 2:0,virtio-net,tap0 -s 3:0,virtio-blk,./linux.img \ -s 4:0,ahci-cd,./somelinux.iso -l com1,stdio -c 4 -m 1024M linuxguest The system will boot and start the installer. After installing a system in the virtual machine, reboot the virtual machine. This will cause bhyve to exit. The instance of the virtual machine needs to be destroyed before it can be started again: &prompt.root; bhyvectl --destroy --vm=linuxguest Now the guest can be started directly from the virtual disk. Load the kernel: &prompt.root; grub-bhyve -m device.map -r hd0,msdos1 -M 1024M linuxguest grub> ls (hd0) (hd0,msdos2) (hd0,msdos1) (cd0) (cd0,msdos1) (host) (lvm/VolGroup-lv_swap) (lvm/VolGroup-lv_root) grub> ls (hd0,msdos1)/ lost+found/ grub/ efi/ System.map-2.6.32-431.el6.x86_64 config-2.6.32-431.el6.x 86_64 symvers-2.6.32-431.el6.x86_64.gz vmlinuz-2.6.32-431.el6.x86_64 initramfs-2.6.32-431.el6.x86_64.img grub> linux (hd0,msdos1)/vmlinuz-2.6.32-431.el6.x86_64 root=/dev/mapper/VolGroup-lv_root grub> initrd (hd0,msdos1)/initramfs-2.6.32-431.el6.x86_64.img grub> boot Boot the virtual machine: &prompt.root; bhyve -A -H -P -s 0:0,hostbridge -s 1:0,lpc -s 2:0,virtio-net,tap0 \ -s 3:0,virtio-blk,./linux.img -l com1,stdio -c 4 -m 1024M linuxguest &linux; will now boot in the virtual machine and eventually present you with the login prompt. Login and use the virtual machine. When you are finished, reboot the virtual machine to exit bhyve. Destroy the virtual machine instance: &prompt.root; bhyvectl --destroy --vm=linuxguest Booting <application>bhyve</application> Virtual Machines with <acronym>UEFI</acronym> Firmware In addition to bhyveload and grub-bhyve, the bhyve hypervisor can also boot virtual machines using the UEFI userspace firmware. This option may support guest operating systems that are not supported by the other loaders. In order to make use of the UEFI support in bhyve, first obtain the UEFI firmware images. This can be done by installing sysutils/bhyve-firmware port or package. With the firmware in place, add the flags to your bhyve command line. The actual bhyve command may look like this: &prompt.root; bhyve -AHP -s 0:0,hostbridge -s 1:0,lpc \ -s 2:0,virtio-net,tap1 -s 3:0,virtio-blk,./disk.img \ -s 4:0,ahci-cd,./install.iso -c 4 -m 1024M \ -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd \ guest sysutils/bhyve-firmware also contains a CSM-enabled firmware, to boot guests with no UEFI support in legacy BIOS mode: &prompt.root; bhyve -AHP -s 0:0,hostbridge -s 1:0,lpc \ -s 2:0,virtio-net,tap1 -s 3:0,virtio-blk,./disk.img \ -s 4:0,ahci-cd,./install.iso -c 4 -m 1024M \ -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI_CSM.fd \ guest Graphical <acronym>UEFI</acronym> Framebuffer for <application>bhyve</application> Guests The UEFI firmware support is particularly useful with predominantly graphical guest operating systems such as Microsoft &windows;. Support for the UEFI-GOP framebuffer may also be enabled with the flags. The framebuffer resolution may be configured with and , and bhyve can be instructed to wait for a VNC connection before booting the guest by adding . The framebuffer may be accessed from the host or over the network via the VNC protocol. The resulting bhyve command would look like this: &prompt.root; bhyve -AHP -s 0:0,hostbridge -s 31:0,lpc \ -s 2:0,virtio-net,tap1 -s 3:0,virtio-blk,./disk.img \ -s 4:0,ahci-cd,./install.iso -c 4 -m 1024M \ -s 29,fbuf,tcp=0.0.0.0:5900,w=800,h=600,wait \ -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd \ guest Note, in BIOS emulation mode, the framebuffer will cease receiving updates once control is passed from firmware to guest operating system. Using <acronym>ZFS</acronym> with <application>bhyve</application> Guests If ZFS is available on the host machine, using ZFS volumes instead of disk image files can provide significant performance benefits for the guest VMs. A ZFS volume can be created by: &prompt.root; zfs create -V16G -o volmode=dev zroot/linuxdisk0 When starting the VM, specify the ZFS volume as the disk drive: &prompt.root; bhyve -A -H -P -s 0:0,hostbridge -s 1:0,lpc -s 2:0,virtio-net,tap0 -s3:0,virtio-blk,/dev/zvol/zroot/linuxdisk0 \ -l com1,stdio -c 4 -m 1024M linuxguest Virtual Machine Consoles It is advantageous to wrap the bhyve console in a session management tool such as sysutils/tmux or sysutils/screen in order to detach and reattach to the console. It is also possible to have the console of bhyve be a null modem device that can be accessed with cu. To do this, load the nmdm kernel module and replace with . The /dev/nmdm devices are created automatically as needed, where each is a pair, corresponding to the two ends of the null modem cable (/dev/nmdm0A and /dev/nmdm0B). See &man.nmdm.4; for more information. &prompt.root; kldload nmdm &prompt.root; bhyve -A -H -P -s 0:0,hostbridge -s 1:0,lpc -s 2:0,virtio-net,tap0 -s 3:0,virtio-blk,./linux.img \ -l com1,/dev/nmdm0A -c 4 -m 1024M linuxguest &prompt.root; cu -l /dev/nmdm0B Connected Ubuntu 13.10 handbook ttyS0 handbook login: Managing Virtual Machines A device node is created in /dev/vmm for each virtual machine. This allows the administrator to easily see a list of the running virtual machines: &prompt.root; ls -al /dev/vmm total 1 dr-xr-xr-x 2 root wheel 512 Mar 17 12:19 ./ dr-xr-xr-x 14 root wheel 512 Mar 17 06:38 ../ crw------- 1 root wheel 0x1a2 Mar 17 12:20 guestname crw------- 1 root wheel 0x19f Mar 17 12:19 linuxguest crw------- 1 root wheel 0x1a1 Mar 17 12:19 otherguest A specified virtual machine can be destroyed using bhyvectl: &prompt.root; bhyvectl --destroy --vm=guestname Persistent Configuration In order to configure the system to start bhyve guests at boot time, the following configurations must be made in the specified files: <filename>/etc/sysctl.conf</filename> net.link.tap.up_on_open=1 <filename>/boot/loader.conf</filename> vmm_load="YES" nmdm_load="YES" if_bridge_load="YES" if_tap_load="YES" <filename>/etc/rc.conf</filename> cloned_interfaces="bridge0 tap0" ifconfig_bridge0="addm igb0 addm tap0" &os; as a &xen;-Host Xen is a GPLv2-licensed type 1 hypervisor for &intel; and &arm; architectures. &os; has included &i386; and &amd; 64-Bit DomU and Amazon EC2 unprivileged domain (virtual machine) support since &os; 8.0 and includes Dom0 control domain (host) support in &os; 11.0. Support for para-virtualized (PV) domains has been removed from &os; 11 in favor of hardware virtualized (HVM) domains, which provides better performance. &xen; is a bare-metal hypervisor, which means that it is the first program loaded after the BIOS. A special privileged guest called the Domain-0 (Dom0 for short) is then started. The Dom0 uses its special privileges to directly access the underlying physical hardware, making it a high-performance solution. It is able to access the disk controllers and network adapters directly. The &xen; management tools to manage and control the &xen; hypervisor are also used by the Dom0 to create, list, and destroy VMs. Dom0 provides virtual disks and networking for unprivileged domains, often called DomU. &xen; Dom0 can be compared to the service console of other hypervisor solutions, while the DomU is where individual guest VMs are run. &xen; can migrate VMs between different &xen; servers. When the two xen hosts share the same underlying storage, the migration can be done without having to shut the VM down first. Instead, the migration is performed live while the DomU is running and there is no need to restart it or plan a downtime. This is useful in maintenance scenarios or upgrade windows to ensure that the services provided by the DomU are still provided. Many more features of &xen; are listed on the Xen Wiki Overview page. Note that not all features are supported on &os; yet. Hardware Requirements for &xen; Dom0 To run the &xen; hypervisor on a host, certain hardware functionality is required. Hardware virtualized domains require Extended Page Table (EPT) and Input/Output Memory Management Unit (IOMMU) support in the host processor. &xen; Dom0 Control Domain Setup The emulators/xen package works with &os; 11 amd64 binary snapshots and equivalent systems built from source. This example assumes VNC output for unprivileged domains which is accessed from a another system using a tool such as net/tightvnc. Install emulators/xen: &prompt.root; pkg install xen Configuration files must be edited to prepare the host for the Dom0 integration. An entry to /etc/sysctl.conf disables the limit on how many pages of memory are allowed to be wired. Otherwise, DomU VMs with higher memory requirements will not run. &prompt.root; sysrc -f /etc/sysctl.conf vm.max_wired=-1 Another memory-related setting involves changing /etc/login.conf, setting the memorylocked option to unlimited. Otherwise, creating DomU domains may fail with Cannot allocate memory errors. After making the change to /etc/login.conf, run cap_mkdb to update the capability database. See for details. &prompt.root; sed -i '' -e 's/memorylocked=64K/memorylocked=unlimited/' /etc/login.conf &prompt.root; cap_mkdb /etc/login.conf Add an entry for the &xen; console to /etc/ttys: &prompt.root; echo 'xc0 "/usr/libexec/getty Pc" xterm on secure' >> /etc/ttys Selecting a &xen; kernel in /boot/loader.conf activates the Dom0. &xen; also requires resources like CPU and memory from the host machine for itself and other DomU domains. How much CPU and memory depends on the individual requirements and hardware capabilities. In this example, 8 GB of memory and 4 virtual CPUs are made available for the Dom0. The serial console is also activated and logging options are defined. &prompt.root; sysrc -f /boot/loader.conf hw.pci.mcfg=0 &prompt.root; sysrc -f /boot/loader.conf xen_kernel="/boot/xen" &prompt.root; sysrc -f /boot/loader.conf xen_cmdline="dom0_mem=8192M dom0_max_vcpus=4 dom0pvh=1 console=com1,vga com1=115200,8n1 guest_loglvl=all loglvl=all" Log files that &xen; creates for the Dom0 and DomU VMs are stored in /var/log/xen. This directory does not exist by default and must be created. &prompt.root; mkdir -p /var/log/xen &prompt.root; chmod 644 /var/log/xen &xen; provides a boot menu to activate and de-activate the hypervisor on demand in /boot/menu.rc.local: &prompt.root; echo "try-include /boot/xen.4th" >> /boot/menu.rc.local Activate the xencommons service during system startup: &prompt.root; sysrc xencommons_enable=yes These settings are enough to start a Dom0-enabled system. However, it lacks network functionality for the DomU machines. To fix that, define a bridged interface with the main NIC of the system which the DomU VMs can use to connect to the network. Replace igb0 with the host network interface name. &prompt.root; sysrc autobridge_interfaces=bridge0 &prompt.root; sysrc autobridge_bridge0=igb0 &prompt.root; sysrc ifconfig_bridge0=SYNCDHCP Restart the host to load the &xen; kernel and start the Dom0. &prompt.root; reboot After successfully booting the &xen; kernel and logging into the system again, the &xen; management tool xl is used to show information about the domains. &prompt.root; xl list Name ID Mem VCPUs State Time(s) Domain-0 0 8192 4 r----- 962.0 The output confirms that the Dom0 (called Domain-0) has the ID 0 and is running. It also has the memory and virtual CPUs that were defined in /boot/loader.conf earlier. More information can be found in the &xen; Documentation. DomU guest VMs can now be created. &xen; DomU Guest VM Configuration Unprivileged domains consist of a configuration file and virtual or physical hard disks. Virtual disk storage for the DomU can be files created by &man.truncate.1; or ZFS volumes as described in . In this example, a 20 GB volume is used. A VM is created with the ZFS volume, a &os; ISO image, 1 GB of RAM and two virtual CPUs. The ISO installation file is retrieved with &man.fetch.1; and saved locally in a file called freebsd.iso. &prompt.root; fetch ftp://ftp.freebsd.org/pub/FreeBSD/releases/ISO-IMAGES/10.3/FreeBSD-10.3-RELEASE-amd64-bootonly.iso -o freebsd.iso A ZFS volume of 20 GB called xendisk0 is created to serve as the disk space for the VM. &prompt.root; zfs create -V20G -o volmode=dev zroot/xendisk0 The new DomU guest VM is defined in a file. Some specific definitions like name, keymap, and VNC connection details are also defined. The following freebsd.cfg contains a minimum DomU configuration for this example: &prompt.root; cat freebsd.cfg builder = "hvm" name = "freebsd" memory = 1024 vcpus = 2 vif = [ 'mac=00:16:3E:74:34:32,bridge=bridge0' ] disk = [ '/dev/zvol/tank/xendisk0,raw,hda,rw', '/root/freebsd.iso,raw,hdc:cdrom,r' ] vnc = 1 vnclisten = "0.0.0.0" serial = "pty" usbdevice = "tablet" These lines are explained in more detail: This defines what kind of virtualization to use. hvm refers to hardware-assisted virtualization or hardware virtual machine. Guest operating systems can run unmodified on CPUs with virtualization extensions, providing nearly the same performance as running on physical hardware. generic is the default value and creates a PV domain. Name of this virtual machine to distinguish it from others running on the same Dom0. Required. Quantity of RAM in megabytes to make available to the VM. This amount is subtracted from the hypervisor's total available memory, not the memory of the Dom0. Number of virtual CPUs available to the guest VM. For best performance, do not create guests with more virtual CPUs than the number of physical CPUs on the host. Virtual network adapter. This is the bridge connected to the network interface of the host. The mac parameter is the MAC address set on the virtual network interface. This parameter is optional, if no MAC is provided &xen; will generate a random one. Full path to the disk, file, or ZFS volume of the disk storage for this VM. Options and multiple disk definitions are separated by commas. Defines the Boot medium from which the initial operating system is installed. In this example, it is the ISO imaged downloaded earlier. Consult the &xen; documentation for other kinds of devices and options to set. Options controlling VNC connectivity to the serial console of the DomU. In order, these are: active VNC support, define IP address on which to listen, device node for the serial console, and the input method for precise positioning of the mouse and other input methods. keymap defines which keymap to use, and is english by default. After the file has been created with all the necessary options, the DomU is created by passing it to xl create as a parameter. &prompt.root; xl create freebsd.cfg Each time the Dom0 is restarted, the configuration file must be passed to xl create again to re-create the DomU. By default, only the Dom0 is created after a reboot, not the individual VMs. The VMs can continue where they left off as they stored the operating system on the virtual disk. The virtual machine configuration can change over time (for example, when adding more memory). The virtual machine configuration files must be properly backed up and kept available to be able to re-create the guest VM when needed. The output of xl list confirms that the DomU has been created. &prompt.root; xl list Name ID Mem VCPUs State Time(s) Domain-0 0 8192 4 r----- 1653.4 freebsd 1 1024 1 -b---- 663.9 To begin the installation of the base operating system, start the VNC client, directing it to the main network address of the host or to the IP address defined on the vnclisten line of freebsd.cfg. After the operating system has been installed, shut down the DomU and disconnect the VNC viewer. Edit freebsd.cfg, removing the line with the cdrom definition or commenting it out by inserting a # character at the beginning of the line. To load this new configuration, it is necessary to remove the old DomU with xl destroy, passing either the name or the id as the parameter. Afterwards, recreate it using the modified freebsd.cfg. &prompt.root; xl destroy freebsd &prompt.root; xl create freebsd.cfg The machine can then be accessed again using the VNC viewer. This time, it will boot from the virtual disk where the operating system has been installed and can be used as a virtual machine.