diff --git a/en_US.ISO8859-1/articles/vm-design/article.sgml b/en_US.ISO8859-1/articles/vm-design/article.sgml
index f932c5eb47..78cfdf1c57 100644
--- a/en_US.ISO8859-1/articles/vm-design/article.sgml
+++ b/en_US.ISO8859-1/articles/vm-design/article.sgml
@@ -1,838 +1,838 @@
-
+
%man;
]>
Design elements of the FreeBSD VM systemMatthewDillondillon@apollo.backplane.comThe title is really just a fancy way of saying that I am going to
attempt to describe the whole VM enchilada, hopefully in a way that
everyone can follow. For the last year I have concentrated on a number
of major kernel subsystems within FreeBSD, with the VM and Swap
- subsystems being the most interesting and NFS being ‘a necessary
- chore’. I rewrote only small portions of the code. In the VM
+ subsystems being the most interesting and NFS being a necessary
+ chore. I rewrote only small portions of the code. In the VM
arena the only major rewrite I have done is to the swap subsystem.
Most of my work was cleanup and maintenance, with only moderate code
rewriting and no major algorithmic adjustments within the VM
subsystem. The bulk of the VM subsystem's theoretical base remains
unchanged and a lot of the credit for the modernization effort in the
last few years belongs to John Dyson and David Greenman. Not being a
historian like Kirk I will not attempt to tag all the various features
with peoples names, since I will invariably get it wrong.This article was originally published in the January 2000 issue of
DaemonNews. This
version of the article may include updates from Matt and other authors
to reflect changes in FreeBSD's VM implementation.IntroductionBefore moving along to the actual design let's spend a little time
on the necessity of maintaining and modernizing any long-living
codebase. In the programming world, algorithms tend to be more
important than code and it is precisely due to BSD's academic roots that
a great deal of attention was paid to algorithm design from the
beginning. More attention paid to the design generally leads to a clean
and flexible codebase that can be fairly easily modified, extended, or
- replaced over time. While BSD is considered an ‘old’
+ replaced over time. While BSD is considered an old
operating system by some people, those of us who work on it tend to view
- it more as a ‘mature’ codebase which has various components
+ it more as a mature codebase which has various components
modified, extended, or replaced with modern code. It has evolved, and
FreeBSD is at the bleeding edge no matter how old some of the code might
be. This is an important distinction to make and one that is
unfortunately lost to many people. The biggest error a programmer can
make is to not learn from history, and this is precisely the error that
many other modern operating systems have made. NT is the best example
of this, and the consequences have been dire. Linux also makes this
mistake to some degree—enough that we BSD folk can make small
jokes about it every once in a while, anyway. Linux's problem is simply
one of a lack of experience and history to compare ideas against, a
problem that is easily and rapidly being addressed by the Linux
community in the same way it has been addressed in the BSD
community—by continuous code development. The NT folk, on the
other hand, repeatedly make the same mistakes solved by Unix decades ago
and then spend years fixing them. Over and over again. They have a
- severe case of ‘not designed here’ and ‘we are always
- right because our marketing department says so’. I have little
+ severe case of not designed here and we are always
+ right because our marketing department says so. I have little
tolerance for anyone who cannot learn from history.Much of the apparent complexity of the FreeBSD design, especially in
the VM/Swap subsystem, is a direct result of having to solve serious
performance issues that occur under various conditions. These issues
are not due to bad algorithmic design but instead rise from
environmental factors. In any direct comparison between platforms,
these issues become most apparent when system resources begin to get
stressed. As I describe FreeBSD's VM/Swap subsystem the reader should
always keep two points in mind. First, the most important aspect of
- performance design is what is known as “Optimizing the Critical
- Path”. It is often the case that performance optimizations add a
+ performance design is what is known as Optimizing the Critical
+ Path. It is often the case that performance optimizations add a
little bloat to the code in order to make the critical path perform
better. Second, a solid, generalized design outperforms a
heavily-optimized design over the long run. While a generalized design
may end up being slower than an heavily-optimized design when they are
first implemented, the generalized design tends to be easier to adapt to
changing conditions and the heavily-optimized design winds up having to
be thrown away. Any codebase that will survive and be maintainable for
years must therefore be designed properly from the beginning even if it
costs some performance. Twenty years ago people were still arguing that
programming in assembly was better than programming in a high-level
language because it produced code that was ten times as fast. Today,
the fallibility of that argument is obvious—as are the parallels
to algorithmic design and code generalization.VM ObjectsThe best way to begin describing the FreeBSD VM system is to look at
it from the perspective of a user-level process. Each user process sees
a single, private, contiguous VM address space containing several types
of memory objects. These objects have various characteristics. Program
code and program data are effectively a single memory-mapped file (the
binary file being run), but program code is read-only while program data
is copy-on-write. Program BSS is just memory allocated and filled with
zeros on demand, called demand zero page fill. Arbitrary files can be
memory-mapped into the address space as well, which is how the shared
library mechanism works. Such mappings can require modifications to
remain private to the process making them. The fork system call adds an
entirely new dimension to the VM management problem on top of the
complexity already given.A program binary data page (which is a basic copy-on-write page)
illustrates the complexity. A program binary contains a preinitialized
data section which is initially mapped directly from the program file.
When a program is loaded into a process's VM space, this area is
initially memory-mapped and backed by the program binary itself,
allowing the VM system to free/reuse the page and later load it back in
from the binary. The moment a process modifies this data, however, the
VM system must make a private copy of the page for that process. Since
the private copy has been modified, the VM system may no longer free it,
because there is no longer any way to restore it later on.You will notice immediately that what was originally a simple file
mapping has become much more complex. Data may be modified on a
page-by-page basis whereas the file mapping encompasses many pages at
once. The complexity further increases when a process forks. When a
process forks, the result is two processes—each with their own
private address spaces, including any modifications made by the original
process prior to the call to fork(). It would be
silly for the VM system to make a complete copy of the data at the time
of the fork() because it is quite possible that at
least one of the two processes will only need to read from that page
from then on, allowing the original page to continue to be used. What
was a private page is made copy-on-write again, since each process
(parent and child) expects their own personal post-fork modifications to
remain private to themselves and not effect the other.FreeBSD manages all of this with a layered VM Object model. The
original binary program file winds up being the lowest VM Object layer.
A copy-on-write layer is pushed on top of that to hold those pages which
had to be copied from the original file. If the program modifies a data
page belonging to the original file the VM system takes a fault and
makes a copy of the page in the higher layer. When a process forks,
additional VM Object layers are pushed on. This might make a little
more sense with a fairly basic example. A fork()
is a common operation for any *BSD system, so this example will consider
a program that starts up, and forks. When the process starts, the VM
system creates an object layer, let's call this A:+---------------+
| A |
+---------------+A pictureA represents the file—pages may be paged in and out of the
file's physical media as necessary. Paging in from the disk is
reasonable for a program, but we really don't want to page back out and
overwrite the executable. The VM system therefore creates a second
layer, B, that will be physically backed by swap space:+---------------+
| B |
+---------------+
| A |
+---------------+On the first write to a page after this, a new page is created in B,
and its contents are initialized from A. All pages in B can be paged in
or out to a swap device. When the program forks, the VM system creates
two new object layers—C1 for the parent, and C2 for the
child—that rest on top of B:+-------+-------+
| C1 | C2 |
+-------+-------+
| B |
+---------------+
| A |
+---------------+In this case, let's say a page in B is modified by the original
parent process. The process will take a copy-on-write fault and
duplicate the page in C1, leaving the original page in B untouched.
Now, let's say the same page in B is modified by the child process. The
process will take a copy-on-write fault and duplicate the page in C2.
The original page in B is now completely hidden since both C1 and C2
have a copy and B could theoretically be destroyed if it does not
represent a 'real' file). However, this sort of optimization is not
trivial to make because it is so fine-grained. FreeBSD does not make
this optimization. Now, suppose (as is often the case) that the child
process does an exec(). Its current address space
is usually replaced by a new address space representing a new file. In
this case, the C2 layer is destroyed:+-------+
| C1 |
+-------+-------+
| B |
+---------------+
| A |
+---------------+In this case, the number of children of B drops to one, and all
accesses to B now go through C1. This means that B and C1 can be
collapsed together. Any pages in B that also exist in C1 are deleted
from B during the collapse. Thus, even though the optimization in the
previous step could not be made, we can recover the dead pages when
either of the processes exit or exec().This model creates a number of potential problems. The first is that
you can wind up with a relatively deep stack of layered VM Objects which
can cost scanning time and memory when you take a fault. Deep
layering can occur when processes fork and then fork again (either
parent or child). The second problem is that you can wind up with dead,
inaccessible pages deep in the stack of VM Objects. In our last example
if both the parent and child processes modify the same page, they both
get their own private copies of the page and the original page in B is
no longer accessible by anyone. That page in B can be freed.FreeBSD solves the deep layering problem with a special optimization
- called the “All Shadowed Case”. This case occurs if either
+ called the All Shadowed Case. This case occurs if either
C1 or C2 take sufficient COW faults to completely shadow all pages in B.
Lets say that C1 achieves this. C1 can now bypass B entirely, so rather
then have C1->B->A and C2->B->A we now have C1->A and C2->B->A. But
look what also happened—now B has only one reference (C2), so we
can collapse B and C2 together. The end result is that B is deleted
entirely and we have C1->A and C2->A. It is often the case that B will
contain a large number of pages and neither C1 nor C2 will be able to
completely overshadow it. If we fork again and create a set of D
layers, however, it is much more likely that one of the D layers will
eventually be able to completely overshadow the much smaller dataset
represented by C1 or C2. The same optimization will work at any point in
the graph and the grand result of this is that even on a heavily forked
machine VM Object stacks tend to not get much deeper then 4. This is
true of both the parent and the children and true whether the parent is
doing the forking or whether the children cascade forks.The dead page problem still exists in the case where C1 or C2 do not
completely overshadow B. Due to our other optimizations this case does
not represent much of a problem and we simply allow the pages to be
dead. If the system runs low on memory it will swap them out, eating a
little swap, but that's it.The advantage to the VM Object model is that
fork() is extremely fast, since no real data
copying need take place. The disadvantage is that you can build a
relatively complex VM Object layering that slows page fault handling
down a little, and you spend memory managing the VM Object structures.
The optimizations FreeBSD makes proves to reduce the problems enough
that they can be ignored, leaving no real disadvantage.SWAP LayersPrivate data pages are initially either copy-on-write or zero-fill
pages. When a change, and therefore a copy, is made, the original
backing object (usually a file) can no longer be used to save a copy of
the page when the VM system needs to reuse it for other purposes. This
is where SWAP comes in. SWAP is allocated to create backing store for
memory that does not otherwise have it. FreeBSD allocates the swap
management structure for a VM Object only when it is actually needed.
However, the swap management structure has had problems
historically.Under FreeBSD 3.x the swap management structure preallocates an
array that encompasses the entire object requiring swap backing
store—even if only a few pages of that object are swap-backed.
This creates a kernel memory fragmentation problem when large objects
are mapped, or processes with large runsizes (RSS) fork. Also, in order
- to keep track of swap space, a ‘list of holes’ is kept in
+ to keep track of swap space, a list of holes is kept in
kernel memory, and this tends to get severely fragmented as well. Since
the 'list of holes' is a linear list, the swap allocation and freeing
performance is a non-optimal O(n)-per-page. It also requires kernel
memory allocations to take place during the swap freeing process, and
that creates low memory deadlock problems. The problem is further
exacerbated by holes created due to the interleaving algorithm. Also,
the swap block map can become fragmented fairly easily resulting in
non-contiguous allocations. Kernel memory must also be allocated on the
fly for additional swap management structures when a swapout occurs. It
is evident that there was plenty of room for improvement.For FreeBSD 4.x, I completely rewrote the swap subsystem. With this
rewrite, swap management structures are allocated through a hash table
rather than a linear array giving them a fixed allocation size and much
finer granularity. Rather then using a linearly linked list to keep
track of swap space reservations, it now uses a bitmap of swap blocks
arranged in a radix tree structure with free-space hinting in the radix
node structures. This effectively makes swap allocation and freeing an
O(1) operation. The entire radix tree bitmap is also preallocated in
order to avoid having to allocate kernel memory during critical low
memory swapping operations. After all, the system tends to swap when it
is low on memory so we should avoid allocating kernel memory at such
times in order to avoid potential deadlocks. Finally, to reduce
fragmentation the radix tree is capable of allocating large contiguous
chunks at once, skipping over smaller fragmented chunks. I did not take
the final step of having an 'allocating hint pointer' that would trundle
through a portion of swap as allocations were made in order to further
guarantee contiguous allocations or at least locality of reference, but
I ensured that such an addition could be made.When to free a pageSince the VM system uses all available memory for disk caching,
there are usually very few truly-free pages. The VM system depends on
being able to properly choose pages which are not in use to reuse for
new allocations. Selecting the optimal pages to free is possibly the
single-most important function any VM system can perform because if it
makes a poor selection, the VM system may be forced to unnecessarily
retrieve pages from disk, seriously degrading system performance.How much overhead are we willing to suffer in the critical path to
avoid freeing the wrong page? Each wrong choice we make will cost us
hundreds of thousands of CPU cycles and a noticeable stall of the
affected processes, so we are willing to endure a significant amount of
overhead in order to be sure that the right page is chosen. This is why
FreeBSD tends to outperform other systems when memory resources become
stressed.The free page determination algorithm is built upon a history of the
use of memory pages. To acquire this history, the system takes advantage
of a page-used bit feature that most hardware page tables have.In any case, the page-used bit is cleared and at some later point
the VM system comes across the page again and sees that the page-used
bit has been set. This indicates that the page is still being actively
used. If the bit is still clear it is an indication that the page is not
being actively used. By testing this bit periodically, a use history (in
the form of a counter) for the physical page is developed. When the VM
system later needs to free up some pages, checking this history becomes
the cornerstone of determining the best candidate page to reuse.What if the hardware has no page-used bit?For those platforms that do not have this feature, the system
actually emulates a page-used bit. It unmaps or protects a page,
forcing a page fault if the page is accessed again. When the page
fault is taken, the system simply marks the page as having been used
and unprotects the page so that it may be used. While taking such page
faults just to determine if a page is being used appears to be an
expensive proposition, it is much less expensive than reusing the page
for some other purpose only to find that a process needs it back and
then have to go to disk.FreeBSD makes use of several page queues to further refine the
selection of pages to reuse as well as to determine when dirty pages
must be flushed to their backing store. Since page tables are dynamic
entities under FreeBSD, it costs virtually nothing to unmap a page from
the address space of any processes using it. When a page candidate has
been chosen based on the page-use counter, this is precisely what is
done. The system must make a distinction between clean pages which can
theoretically be freed up at any time, and dirty pages which must first
be written to their backing store before being reusable. When a page
candidate has been found it is moved to the inactive queue if it is
dirty, or the cache queue if it is clean. A separate algorithm based on
the dirty-to-clean page ratio determines when dirty pages in the
inactive queue must be flushed to disk. Once this is accomplished, the
flushed pages are moved from the inactive queue to the cache queue. At
this point, pages in the cache queue can still be reactivated by a VM
fault at relatively low cost. However, pages in the cache queue are
- considered to be ‘immediately freeable’ and will be reused
+ considered to be immediately freeable and will be reused
in an LRU (least-recently used) fashion when the system needs to
allocate new memory.It is important to note that the FreeBSD VM system attempts to
separate clean and dirty pages for the express reason of avoiding
unnecessary flushes of dirty pages (which eats I/O bandwidth), nor does
it move pages between the various page queues gratuitously when the
memory subsystem is not being stressed. This is why you will see some
systems with very low cache queue counts and high active queue counts
when doing a systat -vm command. As the VM system
becomes more stressed, it makes a greater effort to maintain the various
page queues at the levels determined to be the most effective. An urban
myth has circulated for years that Linux did a better job avoiding
swapouts than FreeBSD, but this in fact is not true. What was actually
occurring was that FreeBSD was proactively paging out unused pages in
order to make room for more disk cache while Linux was keeping unused
pages in core and leaving less memory available for cache and process
pages. I don't know whether this is still true today.Pre-Faulting and Zeroing OptimizationsTaking a VM fault is not expensive if the underlying page is already
in core and can simply be mapped into the process, but it can become
expensive if you take a whole lot of them on a regular basis. A good
example of this is running a program such as &man.ls.1; or &man.ps.1;
over and over again. If the program binary is mapped into memory but
not mapped into the page table, then all the pages that will be accessed
by the program will have to be faulted in every time the program is run.
This is unnecessary when the pages in question are already in the VM
Cache, so FreeBSD will attempt to pre-populate a process's page tables
with those pages that are already in the VM Cache. One thing that
FreeBSD does not yet do is pre-copy-on-write certain pages on exec. For
example, if you run the &man.ls.1; program while running vmstat
1 you will notice that it always takes a certain number of
page faults, even when you run it over and over again. These are
zero-fill faults, not program code faults (which were pre-faulted in
already). Pre-copying pages on exec or fork is an area that could use
more study.A large percentage of page faults that occur are zero-fill faults.
You can usually see this by observing the vmstat -s
output. These occur when a process accesses pages in its BSS area. The
BSS area is expected to be initially zero but the VM system does not
bother to allocate any memory at all until the process actually accesses
it. When a fault occurs the VM system must not only allocate a new page,
it must zero it as well. To optimize the zeroing operation the VM system
has the ability to pre-zero pages and mark them as such, and to request
pre-zeroed pages when zero-fill faults occur. The pre-zeroing occurs
whenever the CPU is idle but the number of pages the system pre-zeros is
limited in order to avoid blowing away the memory caches. This is an
excellent example of adding complexity to the VM system in order to
optimize the critical path.Page Table OptimizationsThe page table optimizations make up the most contentious part of
the FreeBSD VM design and they have shown some strain with the advent of
serious use of mmap(). I think this is actually a
feature of most BSDs though I am not sure when it was first introduced.
There are two major optimizations. The first is that hardware page
tables do not contain persistent state but instead can be thrown away at
any time with only a minor amount of management overhead. The second is
that every active page table entry in the system has a governing
pv_entry structure which is tied into the
vm_page structure. FreeBSD can simply iterate
through those mappings that are known to exist while Linux must check
all page tables that might contain a specific
mapping to see if it does, which can achieve O(n^2) overhead in certain
situations. It is because of this that FreeBSD tends to make better
choices on which pages to reuse or swap when memory is stressed, giving
it better performance under load. However, FreeBSD requires kernel
tuning to accommodate large-shared-address-space situations such as
those that can occur in a news system because it may run out of
pv_entry structures.Both Linux and FreeBSD need work in this area. FreeBSD is trying to
maximize the advantage of a potentially sparse active-mapping model (not
all processes need to map all pages of a shared library, for example),
whereas Linux is trying to simplify its algorithms. FreeBSD generally
has the performance advantage here at the cost of wasting a little extra
memory, but FreeBSD breaks down in the case where a large file is
massively shared across hundreds of processes. Linux, on the other hand,
breaks down in the case where many processes are sparsely-mapping the
same shared library and also runs non-optimally when trying to determine
whether a page can be reused or not.Page ColoringWe'll end with the page coloring optimizations. Page coloring is a
performance optimization designed to ensure that accesses to contiguous
pages in virtual memory make the best use of the processor cache. In
ancient times (i.e. 10+ years ago) processor caches tended to map
virtual memory rather than physical memory. This led to a huge number of
problems including having to clear the cache on every context switch in
some cases, and problems with data aliasing in the cache. Modern
processor caches map physical memory precisely to solve those problems.
This means that two side-by-side pages in a processes address space may
not correspond to two side-by-side pages in the cache. In fact, if you
aren't careful side-by-side pages in virtual memory could wind up using
the same page in the processor cache—leading to cacheable data
being thrown away prematurely and reducing CPU performance. This is true
even with multi-way set-associative caches (though the effect is
mitigated somewhat).FreeBSD's memory allocation code implements page coloring
optimizations, which means that the memory allocation code will attempt
to locate free pages that are contiguous from the point of view of the
cache. For example, if page 16 of physical memory is assigned to page 0
of a process's virtual memory and the cache can hold 4 pages, the page
coloring code will not assign page 20 of physical memory to page 1 of a
process's virtual memory. It would, instead, assign page 21 of physical
memory. The page coloring code attempts to avoid assigning page 20
because this maps over the same cache memory as page 16 and would result
in non-optimal caching. This code adds a significant amount of
complexity to the VM memory allocation subsystem as you can well
imagine, but the result is well worth the effort. Page Coloring makes VM
memory as deterministic as physical memory in regards to cache
performance.ConclusionVirtual memory in modern operating systems must address a number of
different issues efficiently and for many different usage patterns. The
modular and algorithmic approach that BSD has historically taken allows
us to study and understand the current implementation as well as
relatively cleanly replace large sections of the code. There have been a
number of improvements to the FreeBSD VM system in the last several
years, and work is ongoing.Bonus QA session by Allen Briggs
briggs@ninthwonder.com
- What is “the interleaving algorithm” that you
+ What is the interleaving algorithm that you
refer to in your listing of the ills of the FreeBSD 3.x swap
arrangements?FreeBSD uses a fixed swap interleave which defaults to 4. This
means that FreeBSD reserves space for four swap areas even if you
only have one, two, or three. Since swap is interleaved the linear
- address space representing the ‘four swap areas’ will be
+ address space representing the four swap areas will be
fragmented if you don't actually have four swap areas. For
example, if you have two swap areas A and B FreeBSD's address
space representation for that swap area will be interleaved in
blocks of 16 pages:A B C D A B C D A B C D A B C D
- FreeBSD 3.x uses a ‘sequential list of free
- regions’ approach to accounting for the free swap areas.
+ FreeBSD 3.x uses a sequential list of free
+ regions approach to accounting for the free swap areas.
The idea is that large blocks of free linear space can be
represented with a single list node
(kern/subr_rlist.c). But due to the
fragmentation the sequential list winds up being insanely
fragmented. In the above example, completely unused swap will
- have A and B shown as ‘free’ and C and D shown as
- ‘all allocated’. Each A-B sequence requires a list
+ have A and B shown as free and C and D shown as
+ all allocated. Each A-B sequence requires a list
node to account for because C and D are holes, so the list node
cannot be combined with the next A-B sequence.Why do we interleave our swap space instead of just tack swap
areas onto the end and do something fancier? Because it's a whole
lot easier to allocate linear swaths of an address space and have
the result automatically be interleaved across multiple disks than
it is to try to put that sophistication elsewhere.The fragmentation causes other problems. Being a linear list
under 3.x, and having such a huge amount of inherent
fragmentation, allocating and freeing swap winds up being an O(N)
algorithm instead of an O(1) algorithm. Combined with other
factors (heavy swapping) and you start getting into O(N^2) and
O(N^3) levels of overhead, which is bad. The 3.x system may also
need to allocate KVM during a swap operation to create a new list
node which can lead to a deadlock if the system is trying to
pageout pages in a low-memory situation.Under 4.x we do not use a sequential list. Instead we use a
radix tree and bitmaps of swap blocks rather than ranged list
nodes. We take the hit of preallocating all the bitmaps required
for the entire swap area up front but it winds up wasting less
memory due to the use of a bitmap (one bit per block) instead of a
linked list of nodes. The use of a radix tree instead of a
sequential list gives us nearly O(1) performance no matter how
fragmented the tree becomes.I don't get the following:
It is important to note that the FreeBSD VM system attempts
to separate clean and dirty pages for the express reason of
avoiding unnecessary flushes of dirty pages (which eats I/O
bandwidth), nor does it move pages between the various page
queues gratuitously when the memory subsystem is not being
stressed. This is why you will see some systems with very low
cache queue counts and high active queue counts when doing a
systat -vm command.
How is the separation of clean and dirty (inactive) pages
related to the situation where you see low cache queue counts and
high active queue counts in systat -vm? Do the
systat stats roll the active and dirty pages together for the
active queue count?Yes, that is confusing. The relationship is
- “goal” verses “reality”. Our goal is to
+ goal verses reality. Our goal is to
separate the pages but the reality is that if we are not in a
memory crunch, we don't really have to.What this means is that FreeBSD will not try very hard to
separate out dirty pages (inactive queue) from clean pages (cache
queue) when the system is not being stressed, nor will it try to
deactivate pages (active queue -> inactive queue) when the system
is not being stressed, even if they aren't being used. In the &man.ls.1; / vmstat 1 example,
wouldn't some of the page faults be data page faults (COW from
executable file to private page)? I.e., I would expect the page
faults to be some zero-fill and some program data. Or are you
implying that FreeBSD does do pre-COW for the program data?A COW fault can be either zero-fill or program-data. The
mechanism is the same either way because the backing program-data
is almost certainly already in the cache. I am indeed lumping the
two together. FreeBSD does not pre-COW program data or zero-fill,
but it does pre-map pages that exist in its
cache.In your section on page table optimizations, can you give a
little more detail about pv_entry and
vm_page (or should vm_page be
vm_pmap—as in 4.4, cf. pp. 180-181 of
McKusick, Bostic, Karel, Quarterman)? Specifically, what kind of
operation/reaction would require scanning the mappings?How does Linux do in the case where FreeBSD breaks down
(sharing a large file mapping over many processes)?A vm_page represents an (object,index#)
tuple. A pv_entry represents a hardware page
table entry (pte). If you have five processes sharing the same
physical page, and three of those processes's page tables actually
map the page, that page will be represented by a single
vm_page structure and three
pv_entry structures.pv_entry structures only represent pages
mapped by the MMU (one pv_entry represents one
pte). This means that when we need to remove all hardware
references to a vm_page (in order to reuse the
page for something else, page it out, clear it, dirty it, and so
forth) we can simply scan the linked list of
pv_entry's associated with that
vm_page to remove or modify the pte's from
their page tables.Under Linux there is no such linked list. In order to remove
all the hardware page table mappings for a
vm_page linux must index into every VM object
that might have mapped the page. For
example, if you have 50 processes all mapping the same shared
library and want to get rid of page X in that library, you need to
index into the page table for each of those 50 processes even if
only 10 of them have actually mapped the page. So Linux is
trading off the simplicity of its design against performance.
Many VM algorithms which are O(1) or (small N) under FreeBSD wind
up being O(N), O(N^2), or worse under Linux. Since the pte's
representing a particular page in an object tend to be at the same
offset in all the page tables they are mapped in, reducing the
number of accesses into the page tables at the same pte offset
will often avoid blowing away the L1 cache line for that offset,
which can lead to better performance.FreeBSD has added complexity (the pv_entry
scheme) in order to increase performance (to limit page table
accesses to only those pte's that need to be
modified).But FreeBSD has a scaling problem that Linux does not in that
there are a limited number of pv_entry
structures and this causes problems when you have massive sharing
of data. In this case you may run out of
pv_entry structures even though there is plenty
of free memory available. This can be fixed easily enough by
bumping up the number of pv_entry structures in
the kernel config, but we really need to find a better way to do
it.In regards to the memory overhead of a page table verses the
pv_entry scheme: Linux uses
- ‘permanent’ page tables that are not throw away, but
+ permanent page tables that are not throw away, but
does not need a pv_entry for each potentially
- mapped pte. FreeBSD uses ‘throw away’ page tables but
+ mapped pte. FreeBSD uses throw away page tables but
adds in a pv_entry structure for each
actually-mapped pte. I think memory utilization winds up being
about the same, giving FreeBSD an algorithmic advantage with its
ability to throw away page tables at will with very low
overhead.Finally, in the page coloring section, it might help to have a
little more description of what you mean here. I didn't quite
follow it.Do you know how an L1 hardware memory cache works? I'll
explain: Consider a machine with 16MB of main memory but only 128K
of L1 cache. Generally the way this cache works is that each 128K
block of main memory uses the same 128K of
cache. If you access offset 0 in main memory and then offset
offset 128K in main memory you can wind up throwing away the
cached data you read from offset 0!Now, I am simplifying things greatly. What I just described
- is what is called a ‘direct mapped’ hardware memory
+ is what is called a direct mapped hardware memory
cache. Most modern caches are what are called
2-way-set-associative or 4-way-set-associative caches. The
set-associatively allows you to access up to N different memory
regions that overlap the same cache memory without destroying the
previously cached data. But only N.So if I have a 4-way set associative cache I can access offset
0, offset 128K, 256K and offset 384K and still be able to access
offset 0 again and have it come from the L1 cache. If I then
access offset 512K, however, one of the four previously cached
data objects will be thrown away by the cache.It is extremely important…
extremely important for most of a processor's
memory accesses to be able to come from the L1 cache, because the
L1 cache operates at the processor frequency. The moment you have
an L1 cache miss and have to go to the L2 cache or to main memory,
the processor will stall and potentially sit twiddling its fingers
for hundreds of instructions worth of time
waiting for a read from main memory to complete. Main memory (the
dynamic ram you stuff into a computer) is
slow, when compared to the speed of a modern
processor core.Ok, so now onto page coloring: All modern memory caches are
what are known as physical caches. They
cache physical memory addresses, not virtual memory addresses.
This allows the cache to be left alone across a process context
switch, which is very important.But in the Unix world you are dealing with virtual address
spaces, not physical address spaces. Any program you write will
see the virtual address space given to it. The actual
physical pages underlying that virtual
address space are not necessarily physically contiguous! In fact,
you might have two pages that are side by side in a processes
address space which wind up being at offset 0 and offset 128K in
physical memory.A program normally assumes that two side-by-side pages will be
optimally cached. That is, that you can access data objects in
both pages without having them blow away each other's cache entry.
But this is only true if the physical pages underlying the virtual
address space are contiguous (insofar as the cache is
concerned).This is what Page coloring does. Instead of assigning
random physical pages to virtual addresses,
which may result in non-optimal cache performance , Page coloring
assigns reasonably-contiguous physical pages
to virtual addresses. Thus programs can be written under the
assumption that the characteristics of the underlying hardware
cache are the same for their virtual address space as they would
be if the program had been run directly in a physical address
space.
- Note that I say ‘reasonably’ contiguous rather
- than simply ‘contiguous’. From the point of view of a
+ Note that I say reasonably contiguous rather
+ than simply contiguous. From the point of view of a
128K direct mapped cache, the physical address 0 is the same as
the physical address 128K. So two side-by-side pages in your
virtual address space may wind up being offset 128K and offset
132K in physical memory, but could also easily be offset 128K and
offset 4K in physical memory and still retain the same cache
performance characteristics. So page-coloring does
not have to assign truly contiguous pages of
physical memory to contiguous pages of virtual memory, it just
needs to make sure it assigns contiguous pages from the point of
view of cache performance and operation.
diff --git a/en_US.ISO8859-1/books/fdp-primer/sgml-markup/chapter.sgml b/en_US.ISO8859-1/books/fdp-primer/sgml-markup/chapter.sgml
index d99ff7e7dd..39313fb747 100644
--- a/en_US.ISO8859-1/books/fdp-primer/sgml-markup/chapter.sgml
+++ b/en_US.ISO8859-1/books/fdp-primer/sgml-markup/chapter.sgml
@@ -1,2600 +1,2600 @@
SGML MarkupThis chapter describes the two markup languages you will encounter
when you contribute to the FreeBSD documentation project. Each section
describes the markup language, and details the markup that you are likely
to want to use, or that is already in use.These markup languages contain a large number of elements, and it can
be confusing sometimes to know which element to use for a particular
situation. This section goes through the elements you are most likely to
need, and gives examples of how you would use them.This is not an exhaustive list of elements, since
that would just reiterate the documentation for each language. The aim of
this section is to list those elements more likely to be useful to you.
If you have a question about how best to markup a particular piece of
content, please post it to the FreeBSD Documentation Project mailing list
freebsd-doc@FreeBSD.org.Inline vs. blockIn the remainder of this document, when describing elements,
inline means that the element can occur within a
block element, and does not cause a line break. A
block element, by comparison, will cause a line
break (and other processing) when it is encountered.HTMLHTML, the HyperText Markup Language, is the markup language of
choice on the World Wide Web. More information can be found at
<URL:http://www.w3.org/>.HTML is used to markup pages on the FreeBSD web site. It should not
(generally) be used to mark up other documention, since DocBook offers a
far richer set of elements to choose from. Consequently, you will
normally only encounter HTML pages if you are writing for the web
site.HTML has gone through a number of versions, 1, 2, 3.0, 3.2, and the
latest, 4.0 (available in both strict and
loose variants).The HTML DTDs are available from the ports collection in the
textproc/html port. They are automatically
installed as part of the textproc/docproj
port.Formal Public Identifier (FPI)There are a number of HTML FPIs, depending upon the version (also
known as the level) of HTML that you want to declare your document to
be compliant with.The majority of HTML documents on the FreeBSD web site comply with
the loose version of HTML 4.0.PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN"Sectional elementsAn HTML document is normally split in to two sections. The first
section, called the head, contains
meta-information about the document, such as its title, the name of
the author, the parent document, and so on. The second section, the
body, contains the content that will be displayed
to the user.These sections are indicated with head and
body elements respectively. These elements are
contained within the top-level html element.Normal HTML document structure<html>
<head>
<title>The document's title</title>
</head>
<body>
…
</body>
</html>Block elementsHeadingsHTML allows you to denote headings in your document, at up to
six different levels.The largest and most prominent heading is h1,
then h2, continuing down to
h6.The element's content is the text of the heading.h1, h2, etc.Use:First section
This is the heading for the first section
This is the heading for the first sub-section
This is the heading for the second section
]]>Generally, an HTML page should have one first level heading
(h1). This can contain many second level
headings (h2), which can in turn contain many
third level headings. Each
hn element should have
the same element, but one further up the hierarchy, preceeding it.
Leaving gaps in the numbering is to be avoided.Bad ordering of
hn elementsUse:First section
Sub-section
]]>ParagraphsHTML supports a single paragraph element,
p.pUse:This is a paragraph. It can contain just about any
other element.
]]>
Block quotationsA block quotation is an extended quotation from another document
that should not appear within the current paragraph.blockquoteUse:A small excerpt from the US Constitution:
We the People of the United States, in Order to form
a more perfect Union, establish Justice, insure domestic
Tranquility, provide for the common defence, promote the general
Welfare, and secure the Blessings of Liberty to ourselves and our
Posterity, do ordain and establish this Constitution for the
United States of America.
]]>ListsYou can present the user with three types of lists, ordered,
unordered, and definition.Typically, each entry in an ordered list will be numbered, while
each entry in an unordered list will be preceded by a bullet point.
Definition lists are composed of two sections for each entry. The
first section is the term being defined, and the second section is
the definition of the term.Ordered lists are indicated by the ol
element, unordered lists by the ul element, and
definition lists by the dl element.Ordered and unordered lists contain listitems, indicated by the
li element. A listitem can contain textual
content, or it may be further wrapped in one or more
p elements.Definition lists contain definition terms
(dt) and definition descriptions
(dd). A definition term can only contain inline
elements. A definition description can contain other block
elements.ul and olUse:An unordered list. Listitems will probably be
preceeded by bullets.
First item
Second item
Third item
An ordered list, with list items consisting of multiple
paragraphs. Each item (note: not each paragraph) will be
numbered.
This is the first item. It only has one paragraph.
This is the first paragraph of the second item.
This is the second paragraph of the second item.
This is the first and only paragraph of the third
item.
]]>Definition lists with dlUse:
Term 1
Paragraph 1 of definition 1.
Paragraph 2 of definition 1.
Term 2
Paragraph 1 of definition 2.
Term 3
Paragraph 1 of definition 3. Note that the <p>
element is not required in the single paragraph case.
]]>Pre-formatted textYou can indicate that text should be shown to the user exactly
as it is in the file. Typically, this means that the text is shown
in a fixed font, multiple spaces are not merged in to one, and line
breaks in the text are significant.In order to do this, wrap the content in the
pre element.preYou could use pre to mark up an e-mail
message; From: nik@FreeBSD.org
To: freebsd-doc@FreeBSD.org
Subject: New documentation available
There's a new copy of my primer for contributers to the FreeBSD
Documentation Project available at
Comments appreciated.
N]]>TablesMost text-mode browsers (such as Lynx) do not render tables
particularly effectively. If you are relying on the tabular
display of your content, you should consider using alternative
markup to prevent confusion.Mark up tabular information using the table
element. A table consists of one or more table rows
(tr), each containing one or more cells of table
data (td). Each cell can contain other block
elements, such as paragraphs or lists. It can also contain another
table (this nesting can repeat indefinitely). If the cell only
contains one paragraph then you do not need to include the
p element.Simple use of tableUse:This is a simple 2x2 table.
Top left cell
Top right cell
Bottom left cell
Bottom right cell
]]>A cell can span multiple rows and columns. To indicate this,
add the rowspan and/or colspan
attributes, with values indicating the number of rows of columns
that should be spanned.Using rowspanUse:One tall thin cell on the left, two short cells next to
it on the right.
Long and thin
Top cell
Bottom cell
]]>Using colspanUse:One long cell on top, two short cells below it.
Top cell
Bottom left cell
Bottom right cell
]]>Using rowspan and
colspan togetherUse:On a 3x3 grid, the top left block is a 2x2 set of
cells merged in to one. The other cells are normal.
Top left large cell
Top right cell
Middle right cell
Bottom left cell
Bottom middle cell
Bottom right cell
]]>In-line elementsEmphasising informationYou have two levels of emphasis available in HTML,
em and strong.
em is for a normal level of emphasis and
strong indicates stronger emphasis.Typically, em is rendered in italic and
strong is rendered in bold. This is not always
the case, however, and you should not rely on it.em and strongUse:This has been emphasised, while
this has been strongly emphasised.]]>Bold and italicsBecause HTML includes presentational markup, you can also
indicate that particular content should be rendered in bold or
italic. The elements are b and
i respectively.b and iThis is in bold, while this is
in italics.]]>Indicating fixed pitch textIf you have content that should be rendered in a fixed pitch
(typewriter) typeface, use tt (for
- “teletype”).
+ teletype).
ttUse:This document was originally written by
Nik Clayton, who can be reached by e-mail as
nik@FreeBSD.org.]]>Content sizeYou can indicate that content should be shown in a larger or
smaller font. There are three ways of doing this.Use big and small
around the content you wish to change size. These tags can be
nested, so <big><big>This is much
bigger</big></big> is possible.Use font with the size
attribute set to +1 or -1
respectively. This has the same effect as using
big or small. However,
the use of this approach is deprecated.Use font with the size
attribute set to a number between 1 and 7. The default font size
is 3. This approach is deprecated.big, small, and
fontThe following fragments all do the same thing.This text is slightly smaller. But
this text is slightly bigger.
This text is slightly smaller. But
this text is slightly bigger
This text is slightly smaller. But
this text is slightly bigger.
]]>
LinksLinks are also in-line elements.Linking to other documents on the WWWIn order to include a link to another document on the WWW you
must know the URL of the document you want to link to.The link is indicated with a, and the
href attribute contains the URL of the target
document. The content of the element becomes the link, and is
normally indicated to the user in some way (underlining, change of
colour, different mouse cursor when over the link, and so
on).Using <a href="...">Use:More information is available at the
FreeBSD web site.]]>These links will take the user to the top of the chosen
document.Linking to other parts of documentsLinking to a point within another document (or within the same
document) requires that the document author include anchors that you
can link to.Anchors are indicated with a and the
name attribute instead of
href.Using <a name="...">Use:This paragraph can be referenced
in other links with the name para1.]]>To link to a named part of a document, write a normal link to
that document, but include the name of the anchor after a
# symbol.Linking to a named part of another documentAssume that the para1 example resides in a
document called foo.html.More information can be found in the
first paragraph of
foo.html.]]>If you are linking to a named anchor within the same document
then you can omit the document's URL, and just include the name of
the anchor (with the preceeding #).Linking to a named part of the same documentAssume that the para1 example resides in
this documentMore information can be found in the
first paragraph of this
document.]]>DocBookDocBook was designed by the Davenport Group to be
a DTD for writing technical documentation. As such, and unlike LinuxDoc
and HTML, DocBook is very heavily oriented towards markup that
describes what something is, rather than describing
how it should be presented.formal vs. informalSome elements may exist in two forms, formal
and informal. Typically, the formal version of
the element will consist of a title followed by the information
version of the element. The informal version will not have a
title.The DocBook DTD is available from the ports collection in the
textproc/docbook port. It is automatically
installed as part of the textproc/docproj
port.FreeBSD extensionsThe FreeBSD Documentation Project has extended the DocBook DTD by
adding some new elements. These elements serve to make some of the
markup more precise.Where a FreeBSD specific element is listed below it is clearly
marked.Throughout the rest of this document, the term
- “DocBook” is used to mean the FreeBSD extended DocBook
+ DocBook is used to mean the FreeBSD extended DocBook
DTD.There is nothing about these extensions that is FreeBSD
specific, it was just felt that they were useful enhancements for
this particular project. Should anyone from any of the other *nix
camps (NetBSD, OpenBSD, Linux, …) be interested in
collaborating on a standard DocBook extension set, please get in
touch with Nik Clayton nik@FreeBSD.org.The FreeBSD extensions are not (currently) in the ports
collection. They are stored in the FreeBSD CVS tree, as doc/share/sgml/freebsd.dtd.Formal Public Identifier (FPI)In compliance with the DocBook guidelines for writing FPIs for
DocBook customisations, the FPI for the FreeBSD extended DocBook DTD
is;PUBLIC "-//FreeBSD//DTD DocBook V4.1-Based Extension//EN"Document structureDocBook allows you to structure your documentation in several
ways. In the FreeBSD Documentation Project we are using two primary
types of DocBook document: the book and the article.A book is organised into chapters. This is a
mandatory requirement. There may be parts between
the book and the chapter to provide another layer of organisation.
The Handbook is arranged in this way.A chapter may (or may not) contain one or more sections. These
are indicated with the sect1 element. If a section
contains another section then use the sect2
element, and so on, up to sect5.Chapters and sections contain the remainder of the content.An article is simpler than a book, and does not use chapters.
Instead, the content of an article is organised into one or more
sections, using the same sect1 (and
sect2 and so on) elements that are used in
books.Obviously, you should consider the nature of the documentation you
are writing in order to decide whether it is best marked up as a book
or an article. Articles are well suited to information that does not
need to be broken down into several chapters, and that is, relatively
speaking, quite short, at up to 20-25 pages of content. Books are
best suited to information that can be broken up into several
chapters, possibly with appendices and similar content as well.The FreeBSD
tutorials are all marked up as articles, while this
document, the FreeBSD
FAQ, and the FreeBSD Handbook are
all marked up as books.Starting a bookThe content of the book is contained within the
book element. As well as containing structural
markup, this element can contain elements that include additional
information about the book. This is either meta-information, used
for reference purposes, or additional content used to produce a
title page.This additional information should be contained within
bookinfo.Boilerplate book with
bookinfo<book>
<bookinfo>
<title>Your title here</title>
<author>
<firstname>Your first name</firstname>
<surname>Your surname</surname>
<affiliation>
<address><email>Your e-mail address</email></address>
</affiliation>
</author>
<copyright>
<year>1998</year>
<holder role="mailto:your e-mail address">Your name</holder>
</copyright>
<pubdate role="rcs">$Date$</pubdate>
<releaseinfo>$Id$</releaseinfo>
<abstract>
<para>Include an abstract of the book's contents here.</para>
</abstract>
</bookinfo>
…
</book>Starting an articleThe content of the article is contained within the
article element. As well as containing
structural markup, this element can contain elements that include
additional information about the article. This is either
meta-information, used for reference purposes, or additional content
used to produce a title page.This additional information should be contained within
articleinfo.Boilerplate article with
articleinfo<article>
<articleinfo>
<title>Your title here</title>
<author>
<firstname>Your first name</firstname>
<surname>Your surname</surname>
<affiliation>
<address><email>Your e-mail address</email></address>
</affiliation>
</author>
<copyright>
<year>1998</year>
<holder role="mailto:your e-mail address">Your name</holder>
</copyright>
<pubdate role="rcs">$Date$</pubdate>
<releaseinfo>$Id$</releaseinfo>
<abstract>
<para>Include an abstract of the article's contents here.</para>
</abstract>
</articleinfo>
…
</article>Indicating chaptersUse chapter to mark up your chapters. Each
chapter has a mandatory title. Articles do not
contain chapters, they are reserved for books.A simple chapterThe chapter's title
...
]]>A chapter cannot be empty; it must contain elements in addition
to title. If you need to include an empty
chapter then just use an empty paragraph.Empty chaptersThis is an empty chapter
]]>Sections below chaptersIn books, chapters may (but do not need to) be broken up into
sections, subsections, and so on. In articles, sections are the
main structural element, and each article must contain at least one
section. Use the
sectn element. The
n indicates the section number, which
identifies the section level.The first sectn is
sect1. You can have one or more of these in a
chapter. They can contain one or more sect2
elements, and so on, down to sect5.Sections in chaptersA sample chapterSome text in the chapter.First section (1.1)
…
Second section (1.2)First sub-section (1.2.1)First sub-sub-section (1.2.1.1)
…
Second sub-section (1.2.2)
…
]]>This example includes section numbers in the section titles.
You should not do this in your documents. Adding the section
numbers is carried out by the stylesheets (of which more
later), and you do not need to manage them yourself.Subdividing using partsYou can introduce another layer of organisation between
book and chapter with one or
more parts. This cannot be done in an
article.IntroductionOverview
...
What is FreeBSD?
...
History
...
]]>Block elementsParagraphsDocBook supports three types of paragraphs:
formalpara, para, and
simpara.Most of the time you will only need to use
para. formalpara includes a
title element, and simpara
disallows some elements from within para. Stick
with para.paraUse:This is a paragraph. It can contain just about any
other element. ]]>Appearance:This is a paragraph. It can contain just about any other
element.Block quotationsA block quotation is an extended quotation from another document
that should not appear within the current paragraph. You will
probably only need it infrequently.Blockquotes can optionally contain a title and an attribution
(or they can be left untitled and unattributed).blockquoteUse:A small excerpt from the US Constitution;
Preamble to the Constitution of the United StatesCopied from a web site somewhereWe the People of the United States, in Order to form a more perfect
Union, establish Justice, insure domestic Tranquility, provide for the
common defence, promote the general Welfare, and secure the Blessings
of Liberty to ourselves and our Posterity, do ordain and establish this
Constitution for the United States of America.
]]>Appearance:
Preamble to the Constitution of the United StatesCopied from a web site somewhereWe the People of the United States, in Order to form a more
perfect Union, establish Justice, insure domestic Tranquility,
provide for the common defence, promote the general Welfare, and
secure the Blessings of Liberty to ourselves and our Posterity,
do ordain and establish this Constitution for the United States
of America.
Tips, notes, warnings, cautions, important information and
sidebars.You may need to include extra information separate from the
- main body of the text. Typically this is “meta”
+ main body of the text. Typically this is meta
information that the user should be aware of.Depending on the nature of the information, one of
tip, note,
warning, caution, and
important should be used. Alternatively, if the
information is related to the main text but is not one of the above,
use sidebar.The circumstances in which to choose one of these elements over
another is unclear. The DocBook documentation suggests;A Note is for information that should be heeded by all
readers.An Important element is a variation on Note.A Caution is for information regarding possible data loss
or software damage.A Warning is for information regarding possible hardware
damage or injury to life or limb.warningUse:Installing FreeBSD may make you want to delete Windows from your
harddisk.
]]>Installing FreeBSD may make you want to delete Windows from
your harddisk.Lists and proceduresYou will often need to list pieces of information to the user,
or present them with a number of steps that must be carried out in
order to accomplish a particular goal.In order to do this, use itemizedlist,
orderedlist, or
procedureThere are other types of
list element in DocBook, but we're not concerned with those at
the moment.itemizedlist and
orderedlist are similar to their counterparts in
HTML, ul and ol. Each one
consists of one or more listitem elements, and
each listitem contains one or more block
elements. The listitem elements are analagous to
HTML's li tags. However, unlike HTML, they are
required.procedure is slightly different. It consists
of steps, which may in turn consists of more
steps or substeps. Each
step contains block elements.itemizedlist,
orderedlist, and
procedureUse:This is the first itemized item.This is the second itemized item.This is the first ordered item.This is the second ordered item.Do this.Then do this.And now do this.]]>Appearance:This is the first itemized item.This is the second itemized item.This is the first ordered item.This is the second ordered item.Do this.Then do this.And now do this.Showing file samplesIf you want to show a fragment of a file (or perhaps a complete
file) to the user, wrap it in the programlisting
element.White space and line breaks within
programlistingare
significant. In particular, this means that the opening tag should
appear on the same line as the first line of the output, and the
closing tag should appear on the same line as the last line of the
output, otherwise spurious blank lines may be included.programlistingUse:When you have finished, your program should look like
this;
#include <stdio.h>
int
main(void)
{
printf("hello, world\n");
}]]>Notice how the angle brackets in the
#include line need to be referenced by their
entities instead of being included literally.Appearance:When you have finished, your program should look like
this;#include <stdio.h>
int
main(void)
{
printf("hello, world\n");
}CalloutsA callout is a mechanism for referring back to an earlier piece
of text or specific position within an earlier example without
linking to it within the text.To do this, mark areas of interest in your example
(programlisting,
literallayout, or whatever) with the
co element. Each element must have a unique
id assigned to it. After the example include a
calloutlist that refers back to the example and
provides additional commentary.co and
calloutlistWhen you have finished, your program should look like
this;
#include <stdio.h>
int
main(void)
{
printf("hello, world\n");
}Includes the standard IO header file.Specifies that main() returns an
int.The printf() call that writes
hello, world to standard output.]]>Appearance:When you have finished, your program should look like
this;#include <stdio.h>
int
main(void)
{
printf("hello, world\n");
}Includes the standard IO header file.Specifies that main() returns an
int.The printf() call that writes
hello, world to standard output.TablesUnlike HTML, you do not need to use tables for layout purposes,
as the stylesheet handles those issues for you. Instead, just use
tables for marking up tabular data.In general terms (and see the DocBook documentation for more
detail) a table (which can be either formal or informal) consists of
a table element. This contains at least one
tgroup element, which specifies (as an attribute)
the number of columns in this table group. Within the tablegroup
you can then have one thead element, which
contains elements for the table headings (column headings), and one
tbody which contains the body of the
table.Both tgroup and thead
contain row elements, which in turn contain
entry elements. Each entry
element specifies one cell in the table.informaltableUse:This is column head 1This is column head 2Row 1, column 1Row 1, column 2Row 2, column 1Row 2, column 2
]]>Appearance:This is column head 1This is column head 2Row 1, column 1Row 1, column 2Row 2, column 1Row 2, column 2If you don't want a border around the table the
frame attribute can be added to the
informaltable element with a value of
none (i.e., <informaltable
frame="none">).Tables where frame="none"Appearance:This is column head 1This is column head 2Row 1, column 1Row 1, column 2Row 2, column 1Row 2, column 2Examples for the user to followA lot of the time you need to show examples for the user to
follow. Typically, these will consist of dialogs with the computer;
the user types in a command, the user gets a response back, they
type in another command, and so on.A number of distinct elements and entities come in to play
here.screenEverything the user sees in this example will be on the
computer screen, so the next element is
screen.Within screen, white space is
significant.prompt,
&prompt.root; and
&prompt.user;Some of the things the user will be seeing on the screen
are prompts from the computer (either from the OS, command
shell, or application. These should be marked up using
prompt.As a special case, the two shell prompts for the normal
user and the root user have been provided as entities. Every
time you want to indicate the user is at a shell prompt, use
one of &prompt.root; and
&prompt.user; as necessary. They do
not need to be inside prompt.&prompt.root; and
&prompt.user; are FreeBSD
extensions to DocBook, and are not part of the original
DTD.userinputWhen displaying text that the user should type in, wrap it
in userinput tags. It will probably be
displayed differently to the user.screen, prompt, and
userinputUse:&prompt.user; ls -1
foo1
foo2
foo3
&prompt.user; ls -1 | grep foo2
foo2
&prompt.user; suPassword:
&prompt.root; cat foo2
This is the file called 'foo2']]>Appearance:&prompt.user; ls -1
foo1
foo2
foo3
&prompt.user; ls -1 | grep foo2
foo2
&prompt.user; suPassword:
&prompt.root; cat foo2
This is the file called 'foo2'Even though we are displaying the contents of the file
foo2, it is not marked
up as programlisting. Reserve
programlisting for showing fragments of files
outside the context of user actions.In-line elementsEmphasising informationWhen you want to emphasise a particular word or phrase, use
emphasis. This may be presented as italic, or
bold, or might be spoken differently with a text-to-speech
system.There is no way to change the presentation of the emphasis
within your document, no equivalent of HTML's b
and i. If the information you are presenting is
important then consider presenting it in
important rather than
emphasis.emphasisUse:FreeBSD is without doubt the
premiere Unix like operating system for the Intel architecture.]]>Appearance:FreeBSD is without doubt the premiere Unix
like operating system for the Intel architecture.Keys, mouse buttons, and combinationsTo refer to a specific key on the keyboard, use
keycap. To refer to a mouse button, use
mousebutton. And to refer to combinations of key
presses or mouse clicks, wrap them all in
keycombo.keycombo has an attribute called
action, which may be one of
click, double-click,
other, press,
seq, or simul. The last two
values denote whether the keys or buttons should be pressed in
sequence, or simultaneously.The stylesheets automatically add any connecting symbols, such
as +, between the key names, when wrapped in
keycombo.Keys, mouse buttons, and combinationsUse:To switch to the second virtual terminal, press
AltF1.
To exit vi without saving your work, type
Esc:q!.My window manager is configured so that
Altright mouse button is used to move windows.]]>Appearance:To switch to the second virtual terminal, press
AltF1.To exit vi without saving your work, type
Esc:q!.My window manager is configured so that
Altright mouse button is used to move windows.Applications, commands, options, and citesYou will frequently want to refer to both applications and
commands when writing for the Handbook. The distinction between
them is simple: an application is the name for a suite (or possibly
just 1) of programs that fulfil a particular task. A command is the
name of a program that the user can run.In addition, you will occasionally need to list one or more of
the options that a command might take.Finally, you will often want to list a command with its manual
- section number, in the “command(number)” format so
+ section number, in the command(number) format so
common in Unix manuals.Mark up application names with
application.When you want to list a command with its manual section number
(which should be most of the time) the DocBook element is
citerefentry. This will contain a further two
elements, refentrytitle and
manvolnum. The content of
refentrytitle is the name of the command, and the
content of manvolnum is the manual page
section.This can be cumbersome to write, and so a series of general entities
have been created to make this easier. Each entity takes the form
&man.manual-page.manual-section;.The file that contains these entities is in
doc/share/sgml/man-refs.ent, and can be
referred to using this FPI:PUBLIC "-//FreeBSD//ENTITIES DocBook Manual Page Entities//EN"Therefore, the introduction to your documentation will probably
look like this:<!DOCTYPE book PUBLIC "-//FreeBSD//DTD DocBook V4.1-Based Extension//EN" [
<!ENTITY % man PUBLIC "-//FreeBSD//ENTITIES DocBook Manual Page Entities//EN">
%man;
…
]>Use command when you want to include a
- command name “in-line” but present it as something the
+ command name in-line but present it as something the
user should type in.Use option to mark up a command's
options.This can be confusing, and sometimes the choice is not always
clear. Hopefully this example makes it clearer.Applications, commands, and options.Use:Sendmail is the most
widely used Unix mail application.
Sendmail includes the
sendmail8, &man.mailq.8;, and &man.newaliases.8;
programs.One of the command line parameters to sendmail8, , will display the current
status of messages in the mail queue. Check this on the command
line by running sendmail -bp.]]>Appearance:Sendmail is the most widely used
Unix mail application.Sendmail includes the
sendmail8, mailq8, and newaliases8 programs.One of the command line parameters to sendmail8, , will display the current
status of messages in the mail queue. Check this on the command
line by running sendmail -bp.Notice how the
&man.command.section; notation is easier to follow.Files, directories, extensionsWhenever you wish to refer to the name of a file, a directory,
or a file extension, use filename.filenameUse:The SGML source for the Handbook in English can be
found in /usr/doc/en/handbook/. The first
file is called handbook.sgml in that
directory. You should also see a Makefile
and a number of files with a .ent
extension.]]>Appearance:The SGML source for the Handbook in English can be found in
/usr/doc/en/handbook/. The first file is
called handbook.sgml in that directory. You
should also see a Makefile and a number of
files with a .ent extension.DevicesFreeBSD extensionThese elements are part of the FreeBSD extension to DocBook,
and do not exist in the original DocBook DTD.When referring to devices you have two choices. You can either
refer to the device as it appears in /dev, or
you can use the name of the device as it appears in the kernel. For
this latter course, use devicename.Sometimes you will not have a choice. Some devices, such as
networking cards, do not have entries in /dev,
or the entries are markedly different from those entries.devicenameUse:sio is used for serial
communication in FreeBSD. sio manifests
through a number of entries in /dev, including
/dev/ttyd0 and /dev/cuaa0.
By contrast, the networking devices, such as
ed0 do not appear in /dev.
In MS-DOS, the first floppy drive is referred to as
a:. In FreeBSD it is
/dev/fd0.]]>Appearance:sio is used for serial communication
in FreeBSD. sio manifests through a
number of entries in /dev, including
/dev/ttyd0 and
/dev/cuaa0.By contrast, the networking devices, such as
ed0 do not appear in
/dev.In MS-DOS, the first floppy drive is referred to as
a:. In FreeBSD it is
/dev/fd0.Hosts, domains, IP addresses, and so forthFreeBSD extensionThese elements are part of the FreeBSD extension to DocBook,
and do not exist in the original DocBook DTD.You can markup identification information for networked
computers (hosts) in several ways, depending on the nature of the
information. All of them use hostid as the
element, with the role attribute selecting the
type of the marked up information.No role attribute, or
role="hostname"With no role attribute (i.e.,
hostid...hostid the
marked up information is the simple hostname, such as
freefall or wcarchive.
You can explicitly specify this with
role="hostname".role="domainname"The text is a domain name, such as
FreeBSD.org or
ngo.org.uk. There is no hostname
component.role="fqdn"The text is a Fully Qualified Domain Name, with both
hostname and domain name parts.role="ipaddr"The text is an IP address, probably expressed as a dotted
quad.role="ip6addr"The text is an IPv6 address.role="netmask"The text is a network mask, which might be expressed as a
dotted quad, a hexadecimal string, or as a
/ followed by a number.role="mac"The text is an Ethernet MAC address, expressed as a series
of 2 digit hexadecimal numbers separated by colons.hostid and rolesUse:The local machine can always be referred to by the
name localhost, which will have the IP address
127.0.0.1.
The FreeBSD.org domain
contains a number of different hosts, including
freefall.FreeBSD.org and
bento.FreeBSD.org.When adding an IP alias to an interface (using
ifconfig) always use a
netmask of 255.255.255.255
(which can also be expressed as 0xffffffff.The MAC address uniquely identifies every network card
in existence. A typical MAC address looks like 08:00:20:87:ef:d0.]]>Appearance:The local machine can always be referred to by the name
localhost, which will have the IP address 127.0.0.1.The FreeBSD.org domain
contains a number of different hosts, including freefall.FreeBSD.org and bento.FreeBSD.org.When adding an IP alias to an interface (using
ifconfig) always use a
netmask of 255.255.255.255 (which
can also be expressed as 0xffffffff.The MAC address uniquely identifies every network card in
existence. A typical MAC address looks like 08:00:20:87:ef:d0.UsernamesFreeBSD extensionThese elements are part of the FreeBSD extension to DocBook,
and do not exist in the original DocBook DTD.When you need to refer to a specific username, such as
root or bin, use
username.usernameUse:To carry out most system administration functions you
will need to be root.]]>Appearance:To carry out most system administration functions you will
need to be root.Describing MakefilesFreeBSD extensionThese elements are part of the FreeBSD extension to DocBook,
and do not exist in the original DocBook DTD.Two elements exist to describe parts of
Makefiles, maketarget and
makevar.maketarget identifies a build target exported
by a Makefile that can be given as a parameter
to make. makevar identifies a
variable that can be set (in the environment, on the
make command line, or within the
Makefile) to influence the process.maketarget and
makevarUse:Two common targets in a Makefile
are all and clean.
Typically, invoking all will rebuild the
application, and invoking clean will remove
the temporary files (.o for example) created by
the build process.clean may be controlled by a number of
variables, including CLOBBER and
RECURSE.]]>Appearance:Two common targets in a Makefile are
all and
clean.Typically, invoking all will rebuild
the application, and invoking clean will
remove the temporary files (.o for example)
created by the build process.clean may be controlled by a number
of variables, including CLOBBER and
RECURSE.Literal text
- You will often need to include “literal” text in the
+ You will often need to include literal text in the
Handbook. This is text that is excerpted from another file, or
which should be copied from the Handbook into another file
verbatim.Some of the time, programlisting will be
sufficient to denote this text. programlisting
is not always appropriate, particularly when you want to include a
- portion of a file “in-line” with the rest of the
+ portion of a file in-line with the rest of the
paragraph.On these occasions, use literal.literalUse:The maxusers 10 line in the kernel
configuration file determines the size of many system tables, and is
a rough guide to how many simultaneous logins the system will
support.]]>
Appearance:The maxusers 10 line in the kernel
configuration file determines the size of many system tables, and
is a rough guide to how many simultaneous logins the system will
support.Showing items that the user must fill
inThere will often be times when you want to show the user what to
do, or refer to a file, or command line, or similar, where the user
cannot simply copy the examples that you provide, but must instead
include some information themselves.replaceable is designed for this eventuality.
Use it inside other elements to indicate parts
of that element's content that the user must replace.replaceableUse:&prompt.user; man command
]]>Appearance:&prompt.user; man commandreplaceable can be used in many different
elements, including literal. This example also
shows that replaceable should only be wrapped
around the content that the user is meant to
provide. The other content should be left alone.Use:The maxusers n
line in the kernel configuration file determines the size of many system
tables, and is a rough guide to how many simultaneous logins the system will
support.
For a desktop workstation, 32 is a good value
for n.]]>Appearance:The maxusers n
line in the kernel configuration file determines the size of many
system tables, and is a rough guide to how many simultaneous
logins the system will support.For a desktop workstation, 32 is a good
value for n.ImagesImage support in the documentation is currently extremely
experimental. I think the mechanisms described here are unlikely to
change, but that's not guaranteed.You will also need to install the
graphics/ImageMagick port, which is used to
convert between the different image formats. This is a big port,
and most of it is not required. However, while we're working on the
Makefiles and other infrastructure it makes
things easier. This port is not in the
textproc/docproj meta port, you must install it
by hand.The best example of what follows in practice is the
en_US.ISO8859-1/articles/vm-design/ document.
If you're unsure of the description that follows, take a look at the
files in that directory to see how everything hangs togther.
Experiment with creating different formatted versions of the
document to see how the image markup appears in the formatted
output.Image formatsWe currently support two formats for images. The format you
should use will depend on the nature of your image.For images that are primarily vector based, such as network
diagrams, timelines, and similar, use Encapsulated Postscript, and
make sure that your images have the .eps
extension.For bitmaps, such as screen captures, use the Portable Network
Graphic format, and make sure that your images have the
.png extension.These are the only formats in which images
should be committed to the CVS repository.Use the right format for the right image. It is to be expected
that your documentation will have a mix of EPS and PNG images. The
Makefiles ensure that the correct format image
is chosen depending on the output format that you use for your
documentation. Do not commit the same image to the
repository in two different formats.It is anticipated that the Documentation Project will switch to
using the Scalable Vector Graphic (SVG) format for vector images.
However, the current state of SVG capable editing tools makes this
impractical.MarkupThe markup for an image is relatively simple. First, markup a
mediaobject. The mediaobject
can contain other, more specific objects. We are concerned with
two, the imageobject and the
textobject.You should include one imageobject, and two
textobject elements. The
imageobject will point to the name of the image
file that will be used (without the extension). The
textobject elements contain information that will
be presented to the user as well as, or instead of, the
image.There are two circumstances where this can happen.When the reader is viewing the documentation in HTML. In
this case, each image will need to have associated alternate
text to show the user, typically whilst the image is loading, or
if they hover the mouse pointer over the image.When the reader is viewing the documentation in plain text.
In this case, each image should have an ASCII art equivalent to
show the user.An example will probably make things easier to understand.
Suppose you have an image, called fig1, that
you want to include in the document. This image is of a rectangle
with an A inside it. The markup for this would be as
follows.<mediaobject>
<imageobject>
<imagedata fileref="fig1">
</imageobject>
<textobject>
<literallayout class="monospaced">+---------------+
| A |
+---------------+</literallayout>
</textobject>
<textobject>
<phrase>A picture</phrase>
</textobject>
</mediaobject>Include an imagedata element inside the
imageobject element. The
fileref attribute should contain the filename
of the image to include, without the extension. The stylesheets
will work out which extension should be added to the filename
automatically.The first textobject should contain a
literallayout element, where the
class attribute is set to
monospaced. This is your opportunity to
demonstrate your ASCII art skills. This content will be used if
the document is converted to plain text.Notice how the first and last lines of the content of the
literallayout element butt up next to the
element's tags. This ensures no extraneous white space is
included.The second textobject should contain a
single phrase element. The contents of this
will become the alt attribute for the image
when this document is converted to HTML.Makefile entriesYour images must be listed in the
Makefile in the IMAGES
variable. This variable should contain the name of all your
source images. For example, if you have
created three figures, fig1.eps,
fig2.png, fig3.png, then
your Makefile should have lines like this in
it.…
IMAGES= fig1.eps fig2.png fig3.png
…or…
IMAGES= fig1.eps
IMAGES+= fig2.png
IMAGES+= fig3.png
…Again, the Makefile will work out the
complete list of images it needs to build your source document, you
only need to list the image files you
provided.Images and chapters in subdirectoriesYou must be careful when you separate your documentation in to
smaller files (see ) in
different directories.Suppose you have a book with three chapters, and the chapters
are stored in their own directories, called
chapter1/chapter.sgml,
chapter2/chapter.sgml, and
chapter3/chapter.sgml. If each chapter has
images associated with it, I suggest you place those images in each
chapter's subdirectory (chapter1/,
chapter2/, and
chapter3/).However, if you do this you must include the directory names in
the IMAGES variable in the
Makefile, and you must
include the directory name in the imagedata
element in your document.For example, if you have chapter1/fig1.png,
then chapter1/chapter.sgml should
contain<mediaobject>
<imageobject>
<imagedata fileref="chapter1/fig1">
</imageobject>
…
</mediaobject>The directory name must be included in the
fileref attributeThe Makefile must contain…
IMAGES= chapter1/fig1.png
…Then everything should just work.LinksLinks are also in-line elements.Linking to other parts of the same documentLinking within the same document requires you to specify
where you are linking from (i.e., the text the user will click, or
otherwise indicate, as the source of the link) and where you are
linking to (the link's destination).Each element within DocBook has an attribute called
id. You can place text in this attribute to
uniquely name the element it is attached to.This value will be used when you specify the link
source.Normally, you will only be linking to chapters or sections, so
you would add the id attribute to these
elements.id on chapters and sectionsIntroductionThis is the introduction. It contains a subsection,
which is identified as well.Sub-sect 1This is the subsection.
]]>Obviously, you should use more descriptive values. The values
must be unique within the document (i.e., not just the file, but the
document the file might be included in as well). Notice how the
id for the subsection is constructed by appending
text to the id of the chapter. This helps to
ensure that they are unique.If you want to allow the user to jump into a specific portion of
the document (possibly in the middle of a paragraph or an example),
use anchor. This element has no content, but
takes an id attribute.anchorThis paragraph has an embedded
link target in it. It won't show up in
the document.]]>When you want to provide the user with a link they can activate
(probably by clicking) to go to a section of the document that has
an id attribute, you can use either
xref or link.Both of these elements have a linkend
attribute. The value of this attribute should be the value that you
have used in a id attribute (it does not matter
if that value has not yet occurred in your document; this will work
for forward links as well as backward links).If you use xref then you have no control over
the text of the link. It will be generated for you.Using xrefAssume that this fragment appears somewhere in a document that
includes the id example;More information can be found
in .
More specific information can be found
in .]]>The text of the link will be generated automatically, and will
look like (emphasised text indicates the text
that will be the link);
More information can be found in Chapter
One.More specific information can be found in the
section called Sub-sect 1.
Notice how the text from the link is derived from the section
title or the chapter number.This means that you cannot use
xref to link to an id
attribute on an anchor element. The
anchor has no content, so the
xref cannot generate the text for the
link.If you want to control the text of the link then use
link. This element wraps content, and the
content will be used for the link.Using linkAssume that this fragment appears somewhere in a document that
includes the id example.More information can be found in
the first chapter.
More specific information can be found in
this section.]]>This will generate the following
(emphasised text indicates the text that will
be the link);
More information can be found in the first
chapter.More specific information can be found in
this section.
That last one is a bad example. Never use words like
- “this” or “here” as the source for the
+ this or here as the source for the
link. The reader will need to hunt around the surrounding context
to see where the link is actually taking them.You can use link to
include a link to an id on an
anchor element, since the
link content defines the text that will be used
for the link.Linking to documents on the WWWLinking to external documents is much simpler, as long as you
know the URL of the document you want to link to. Use
ulink. The url attribute is
the URL of the page that the link points to, and the content of the
element is the text that will be displayed for the user to
activate.ulinkUse:Of course, you could stop reading this document and
go to the FreeBSD
home page instead.]]>Appearance:Of course, you could stop reading this document and go to the
FreeBSD home page
instead.
diff --git a/en_US.ISO8859-1/books/fdp-primer/sgml-primer/chapter.sgml b/en_US.ISO8859-1/books/fdp-primer/sgml-primer/chapter.sgml
index 9e851bee54..cdac8cd636 100644
--- a/en_US.ISO8859-1/books/fdp-primer/sgml-primer/chapter.sgml
+++ b/en_US.ISO8859-1/books/fdp-primer/sgml-primer/chapter.sgml
@@ -1,1556 +1,1556 @@
SGML PrimerThe majority of FDP documentation is written in applications of
SGML. This chapter explains exactly what that means, how to read
and understand the source to the documentation, and the sort of SGML
tricks you will see used in the documentation.Portions of this section were inspired by Mark Galassi's Get Going With DocBook.OverviewWay back when, electronic text was simple to deal with. Admittedly,
you had to know which character set your document was written in (ASCII,
EBCDIC, or one of a number of others) but that was about it. Text was
text, and what you saw really was what you got. No frills, no
formatting, no intelligence.Inevitably, this was not enough. Once you have text in a
machine-usable format, you expect machines to be able to use it and
manipulate it intelligently. You would like to indicate that certain
phrases should be emphasised, or added to a glossary, or be hyperlinks.
- You might want filenames to be shown in a “typewriter” style
- font for viewing on screen, but as “italics” when printed,
+ You might want filenames to be shown in a typewriter style
+ font for viewing on screen, but as italics when printed,
or any of a myriad of other options for presentation.It was once hoped that Artificial Intelligence (AI) would make this
easy. Your computer would read in the document and automatically
identify key phrases, filenames, text that the reader should type in,
examples, and more. Unfortunately, real life has not happened quite
like that, and our computers require some assistance before they can
meaningfully process our text.More precisely, they need help identifying what is what. You or I
can look at
To remove /tmp/foo use &man.rm.1;.&prompt.user; rm /tmp/foo
and easily see which parts are filenames, which are commands to be typed
in, which parts are references to manual pages, and so on. But the
computer processing the document cannot. For this we need
markup.
- “Markup” is commonly used to describe “adding
- value” or “increasing cost”. The term takes on both
+ Markup is commonly used to describe adding
+ value or increasing cost. The term takes on both
these meanings when applied to text. Markup is additional text included
in the document, distinguished from the document's content in some way,
so that programs that process the document can read the markup and use
it when making decisions about the document. Editors can hide the
markup from the user, so the user is not distracted by it.The extra information stored in the markup adds
value to the document. Adding the markup to the document
must typically be done by a person—after all, if computers could
recognise the text sufficiently well to add the markup then there would
be no need to add it in the first place. This increases the
cost (i.e., the effort required) to create the
document.The previous example is actually represented in this document like
this;To remove /tmp/foo use &man.rm.1;.&prompt.user; rm /tmp/foo]]>
As you can see, the markup is clearly separate from the
content.Obviously, if you are going to use markup you need to define what
your markup means, and how it should be interpreted. You will need a
markup language that you can follow when marking up your
documents.Of course, one markup language might not be enough. A markup
language for technical documentation has very different requirements
than a markup language that was to be used for cookery recipes. This,
in turn, would be very different from a markup language used to describe
poetry. What you really need is a first language that you use to write
these other markup languages. A meta markup
language.This is exactly what the Standard Generalised Markup Language (SGML)
is. Many markup languages have been written in SGML, including the two
most used by the FDP, HTML and DocBook.Each language definition is more properly called a Document Type
Definition (DTD). The DTD specifies the name of the elements that can
be used, what order they appear in (and whether some markup can be used
inside other markup) and related information. A DTD is sometimes
referred to as an application of SGML.A DTD is a complete
specification of all the elements that are allowed to appear, the order
in which they should appear, which elements are mandatory, which are
optional, and so forth. This makes it possible to write an SGML
parser which reads in both the DTD and a document
which claims to conform to the DTD. The parser can then confirm whether
or not all the elements required by the DTD are in the document in the
right order, and whether there are any errors in the markup. This is
normally referred to as validating the document.This processing simply confirms that the choice of elements, their
ordering, and so on, conforms to that listed in the DTD. It does
not check that you have used
appropriate markup for the content. If you were
to try and mark up all the filenames in your document as function
names, the parser would not flag this as an error (assuming, of
course, that your DTD defines elements for filenames and functions,
and that they are allowed to appear in the same place).It is likely that most of your contributions to the Documentation
Project will consist of content marked up in either HTML or DocBook,
rather than alterations to the DTDs. For this reason this book will
not touch on how to write a DTD.Elements, tags, and attributesAll the DTDs written in SGML share certain characteristics. This is
hardly surprising, as the philosophy behind SGML will inevitably show
through. One of the most obvious manifestations of this philisophy is
that of content and
elements.Your documentation (whether it is a single web page, or a lengthy
book) is considered to consist of content. This content is then divided
(and further subdivided) into elements. The purpose of adding markup is
to name and identify the boundaries of these elements for further
processing.For example, consider a typical book. At the very top level, the
- book is itself an element. This “book” element obviously
+ book is itself an element. This book element obviously
contains chapters, which can be considered to be elements in their own
right. Each chapter will contain more elements, such as paragraphs,
quotations, and footnotes. Each paragraph might contain further
elements, identifying content that was direct speech, or the name of a
character in the story.
- You might like to think of this as “chunking” content.
+ You might like to think of this as chunking content.
At the very top level you have one chunk, the book. Look a little
deeper, and you have more chunks, the individual chapters. These are
chunked further into paragraphs, footnotes, character names, and so
on.Notice how you can make this differentation between different
elements of the content without resorting to any SGML terms. It really
is surprisingly straightforward. You could do this with a highlighter
pen and a printout of the book, using different colours to indicate
different chunks of content.Of course, we do not have an electronic highlighter pen, so we need
some other way of indicating which element each piece of content belongs
to. In languages written in SGML (HTML, DocBook, et al) this is done by
means of tags.A tag is used to identify where a particular element starts, and
where the element ends. The tag is not part of the element
itself. Because each DTD was normally written to mark up
specific types of information, each one will recognise different
elements, and will therefore have different names for the tags.For an element called element-name the
start tag will normally look like
<element-name>. The
corresponding closing tag for this element is
</element-name>.Using an element (start and end tags)HTML has an element for indicating that the content enclosed by
the element is a paragraph, called p. This
element has both start and end tags.This is a paragraph. It starts with the start tag for
the 'p' element, and it will end with the end tag for the 'p'
element.
This is another paragraph. But this one is much shorter.
]]>Not all elements require an end tag. Some elements have no content.
For example, in HTML you can indicate that you want a horizontal line to
appear in the document. Obviously, this line has no content, so just
the start tag is required for this element.Using an element (start tag only)HTML has an element for indicating a horizontal rule, called
hr. This element does not wrap content, so only
has a start tag.This is a paragraph.
This is another paragraph. A horizontal rule separates this
from the previous paragraph.
]]>If it is not obvious by now, elements can contain other elements.
In the book example earlier, the book element contained all the chapter
elements, which in turn contained all the paragraph elements, and so
on.Elements within elements; emThis is a simple paragraph where some
of the words have been emphasised.]]>The DTD will specify the rules detailing which elements can contain
other elements, and exactly what they can contain.People often confuse the terms tags and elements, and use the
terms as if they were interchangeable. They are not.An element is a conceptual part of your document. An element has
a defined start and end. The tags mark where the element starts and
end.When this document (or anyone else knowledgable about SGML) refers
- to “the <p> tag” they mean the literal text
+ to the <p> tag they mean the literal text
consisting of the three characters <,
p, and >. But the phrase
- “the <p> element” refers to the whole
+ the <p> element refers to the whole
element.This distinction is very subtle. But keep it
in mind.Elements can have attributes. An attribute has a name and a value,
and is used for adding extra information to the element. This might be
information that indicates how the content should be rendered, or might
be something that uniquely identifies that occurence of the element, or
it might be something else.An element's attributes are written inside the
start tag for that element, and take the form
attribute-name="attribute-value".In sufficiently recent versions of HTML, the p
element has an attribute called align, which suggests
an alignment (justification) for the paragraph to the program displaying
the HTML.The align attribute can take one of four defined
values, left, center,
right and justify. If the
attribute is not specified then the default is
left.Using an element with an attributeThe inclusion of the align attribute
on this paragraph was superfluous, since the default is left.
This may appear in the center.
]]>Some attributes will only take specific values, such as
left or justify. Others will
allow you to enter anything you want. If you need to include quotes
(") within an attribute then use single quotes around
the attribute value.Single quotes around attributesI'm on the right!]]>Sometimes you do not need to use quotes around attribute values at
all. However, the rules for doing this are subtle, and it is far
simpler just to always quote your attribute
values.For you to do…In order to run the examples in this document you will need to
install some software on your system and ensure that an environment
variable is set correctly.Download and install textproc/docproj
from the FreeBSD ports system. This is a
meta-port that should download and install
all of the programs and supporting files that are used by the
Documentation Project.Add lines to your shell startup files to set
SGML_CATALOG_FILES..profile, for &man.sh.1; and
&man.bash.1; usersSGML_ROOT=/usr/local/share/sgml
SGML_CATALOG_FILES=${SGML_ROOT}/jade/catalog
SGML_CATALOG_FILES=${SGML_ROOT}/iso8879/catalog:$SGML_CATALOG_FILES
SGML_CATALOG_FILES=${SGML_ROOT}/html/catalog:$SGML_CATALOG_FILES
SGML_CATALOG_FILES=${SGML_ROOT}/docbook/4.1/catalog:$SGML_CATALOG_FILES
export SGML_CATALOG_FILES.login, for &man.csh.1; and
&man.tcsh.1; userssetenv SGML_ROOT /usr/local/share/sgml
setenv SGML_CATALOG_FILES ${SGML_ROOT}/jade/catalog
setenv SGML_CATALOG_FILES ${SGML_ROOT}/iso8879/catalog:$SGML_CATALOG_FILES
setenv SGML_CATALOG_FILES ${SGML_ROOT}/html/catalog:$SGML_CATALOG_FILES
setenv SGML_CATALOG_FILES ${SGML_ROOT}/docbook/4.1/catalog:$SGML_CATALOG_FILESThen either log out, and log back in again, or run those
commands from the command line to set the variable values.Create example.sgml, and enter the
following text;An example HTML file
This is a paragraph containing some text.
This paragraph contains some more text.
This paragraph might be right-justified.
]]>Try and validate this file using an SGML parser.Part of textproc/docproj is the
&man.nsgmls.1; validating
parser. Normally, &man.nsgmls.1; reads in a document
marked up according to an SGML DTD and returns a copy of the
document's Element Structure Information Set (ESIS, but that is
not important right now).However, when &man.nsgmls.1; is given the
parameter, &man.nsgmls.1; will suppress its normal output, and
just print error messages. This makes it a useful way to check to
see if your document is valid or not.Use &man.nsgmls.1; to check that your document is
valid;&prompt.user; nsgmls -s example.sgmlAs you will see, &man.nsgmls.1; returns without displaying any
output. This means that your document validated
successfully.See what happens when required elements are omitted. Try
removing the title and
/title tags, and re-run the validation.&prompt.user; nsgmls -s example.sgml
nsgmls:example.sgml:5:4:E: character data is not allowed here
nsgmls:example.sgml:6:8:E: end tag for "HEAD" which is not finishedThe error output from &man.nsgmls.1; is organised into
colon-separated groups, or columns.ColumnMeaning1The name of the program generating the error. This
will always be nsgmls.2The name of the file that contains the error.3Line number where the error appears.4Column number where the error appears.5A one letter code indicating the nature of the
message. I indicates an informational
message, W is for warnings, and
E is for errorsIt is not always the fifth column either.
nsgmls -sv displays
nsgmls:I: SP version "1.3"
(depending on the installed version). As you can see,
this is an informational message., and X is for
cross-references. As you can see, these messages are
errors.6The text of the error message.Simply omitting the title tags has
generated 2 different errors.The first error indicates that content (in this case,
characters, rather than the start tag for an element) has occured
where the SGML parser was expecting something else. In this case,
the parser was expecting to see one of the start tags for elements
that are valid inside head (such as
title).The second error is because head elements
must contain a title
element. Because it does not &man.nsgmls.1; considers that the
element has not been properly finished. However, the closing tag
indicates that the element has been closed before it has been
finished.Put the title element back in.The DOCTYPE declarationThe beginning of each document that you write must specify the name
of the DTD that the document conforms to. This is so that SGML parsers
can determine the DTD and ensure that the document does conform to
it.This information is generally expressed on one line, in the DOCTYPE
declaration.A typical declaration for a document written to conform with version
4.0 of the HTML DTD looks like this;]]>That line contains a number of different components.<!Is the indicator that indicates that this
is an SGML declaration. This line is declaring the document type.
DOCTYPEShows that this is an SGML declaration for the document
type.htmlNames the first element that
will appear in the document.PUBLIC "-//W3C//DTD HTML 4.0//EN"Lists the Formal Public Identifier (FPI)Formal Public Identifier
for the DTD that this
document conforms to. Your SGML parser will use this to find the
correct DTD when processing this document.PUBLIC is not a part of the FPI, but
indicates to the SGML processor how to find the DTD referenced in
the FPI. Other ways of telling the SGML parser how to find the
DTD are shown later.>Returns to the document.Formal Public Identifiers (FPIs)Formal Public IdentifierYou don't need to know this, but it's useful background, and
might help you debug problems when your SGML processor can't locate
the DTD you are using.FPIs must follow a specific syntax. This syntax is as
follows;"Owner//KeywordDescription//Language"OwnerThis indicates the owner of the FPI.
- If this string starts with “ISO” then this is an
+ If this string starts with ISO then this is an
ISO owned FPI. For example, the FPI "ISO
8879:1986//ENTITIES Greek Symbols//EN" lists
ISO 8879:1986 as being the owner for the set
of entities for greek symbols. ISO 8879:1986 is the ISO number
for the SGML standard.Otherwise, this string will either look like
-//Owner or
+//Owner (notice
the only difference is the leading + or
-).If the string starts with - then the
owner information is unregistered, with a +
it identifies it as being registered.ISO 9070:1991 defines how registered names are generated; it
might be derived from the number of an ISO publication, an ISBN
code, or an organisation code assigned according to ISO 6523.
In addition, a registration authority could be created in order
to assign registered names. The ISO council delegated this to
the American National Standards Institute (ANSI).Because the FreeBSD Project hasn't been registered the
owner string is -//FreeBSD. And as you can
see, the W3C are not a registered owner either.KeywordThere are several keywords that indicate the type of
information in the file. Some of the most common keywords are
DTD, ELEMENT,
ENTITIES, and TEXT.
DTD is used only for DTD files,
ELEMENT is usually used for DTD fragments
that contain only entity or element declarations.
TEXT is used for SGML content (text and
tags).DescriptionAny description you want to supply for the contents of this
file. This may include version numbers or any short text that
is meaningful to you and unique for the SGML system.LanguageThis is an ISO two-character code that identifies the native
language for the file. EN is used for
English.catalog filesIf you use the syntax above and try and process this document
using an SGML processor, the processor will need to have some way of
turning the FPI into the name of the file on your computer that
contains the DTD.In order to do this it can use a catalog file. A catalog file
(typically called catalog) contains lines that
map FPIs to filenames. For example, if the catalog file contained
the line;PUBLIC "-//W3C//DTD HTML 4.0//EN" "4.0/strict.dtd"The SGML processor would know to look up the DTD from
strict.dtd in the 4.0
subdirectory of whichever directory held the
catalog file that contained that line.Look at the contents of
/usr/local/share/sgml/html/catalog. This is
the catalog file for the HTML DTDs that will have been installed as
part of the textproc/docproj port.SGML_CATALOG_FILESIn order to locate a catalog file, your
SGML processor will need to know where to look. Many of them
feature command line parameters for specifying the path to one or
more catalogs.In addition, you can set SGML_CATALOG_FILES to
point to the files. This environment variable should consist of a
colon-separated list of catalog files (including their full
path).Typically, you will want to include the following files;/usr/local/share/sgml/docbook/4.1/catalog/usr/local/share/sgml/html/catalog/usr/local/share/sgml/iso8879/catalog/usr/local/share/sgml/jade/catalogYou should already have done
this.Alternatives to FPIsInstead of using an FPI to indicate the DTD that the document
conforms to (and therefore, which file on the system contains the DTD)
you can explicitly specify the name of the file.The syntax for this is slightly different:]]>The SYSTEM keyword indicates that the SGML
processor should locate the DTD in a system specific fashion. This
typically (but not always) means the DTD will be provided as a
filename.Using FPIs is preferred for reasons of portability. You don't
want to have to ship a copy of the DTD around with your document, and
if you used the SYSTEM identifier then everyone
would need to keep their DTDs in the same place.Escaping back to SGMLEarlier in this primer I said that SGML is only used when writing a
DTD. This is not strictly true. There is certain SGML syntax that you
will want to be able to use within your documents. For example,
comments can be included in your document, and will be ignored by the
parser. Comments are entered using SGML syntax. Other uses for SGML
syntax in your document will be shown later too.Obviously, you need some way of indicating to the SGML processor
that the following content is not elements within the document, but is
SGML that the parser should act upon.These sections are marked by <! ... > in
your document. Everything between these delimiters is SGML syntax as
you might find within a DTD.As you may just have realised, the DOCTYPE declaration
is an example of SGML syntax that you need to include in your
document…CommentsComments are an SGML construction, and are normally only valid
inside a DTD. However, as
shows, it is possible to use SGML syntax within your document.The delimiter for SGML comments is the string
- “--”. The first occurence of this string
+ --. The first occurence of this string
opens a comment, and the second closes it.SGML generic comment<!-- test comment -->
]]>Use 2 dashesThere is a problem with producing the Postscript and PDF versions
of this document. The above example probably shows just one hyphen
symbol, - after the <! and
before the >.You must use two -,
not one. The Postscript and PDF versions have
translated the two - in the original to a longer,
more professional em-dash, and broken this
example in the process.The HTML, plain text, and RTF versions of this document are not
affected.
]]>
If you have used HTML before you may have been shown different rules
for comments. In particular, you may think that the string
<!-- opens a comment, and it is only closed by
-->.This is not the case. A lot of web browsers
have broken HTML parsers, and will accept that as valid. However, the
SGML parsers used by the Documentation Project are much stricter, and
will reject documents that make that error.Errorneous SGML comments]]>The SGML parser will treat this as though it were actually;<!THIS IS OUTSIDE THE COMMENT>This is not valid SGML, and may give confusing error
messages.]]>As the example suggests, do not write
comments like that.]]>That is a (slightly) better approach, but it still potentially
confusing to people new to SGML.For you to do…Add some comments to example.sgml, and
check that the file still validates using &man.nsgmls.1;Add some invalid comments to
example.sgml, and see the error messages that
&man.nsgmls.1; gives when it encounters an invalid comment.EntitiesEntities are a mechanism for assigning names to chunks of content.
As an SGML parser processes your document, any entities it finds are
replaced by the content of the entity.This is a good way to have re-usable, easily changeable chunks of
content in your SGML documents. It is also the only way to include one
marked up file inside another using SGML.There are two types of entities which can be used in two different
situations; general entities and
parameter entities.General EntitiesYou cannot use general entities in an SGML context (although you
define them in one). They can only be used in your document.
Contrast this with parameter
entities.Each general entity has a name. When you want to reference a
general entity (and therefore include whatever text it represents in
your document), you write
&entity-name;. For
example, suppose you had an entity called
current.version which expanded to the current
version number of your product. You could write;The current version of our product is
¤t.version;.]]>When the version number changes you can simply change the
definition of the value of the general entity and reprocess your
document.You can also use general entities to enter characters that you
could not otherwise include in an SGML document. For example, <
and & cannot normally appear in an SGML document. When the SGML
parser sees the < symbol it assumes that a tag (either a start tag
or an end tag) is about to appear, and when it sees the & symbol
it assumes the next text will be the name of an entity.Fortunately, you can use the two general entities < and
& whenever you need to include one or other of these A general entity can only be defined within an SGML context.
Typically, this is done immediately after the DOCTYPE
declaration.Defining general entities
]>]]>Notice how the DOCTYPE declaration has been extended by adding a
square bracket at the end of the first line. The two entities are
then defined over the next two lines, before the square bracket is
closed, and then the DOCTYPE declaration is closed.The square brackets are necessary to indicate that we are
extending the DTD indicated by the DOCTYPE declaration.Parameter entitiesLike general
entities, parameter entities are used to assign names to
reusable chunks of text. However, where as general entities can only
be used within your document, parameter entities can only be used
within an SGML
context.Parameter entities are defined in a similar way to general
entities. However, instead of using
&entity-name; to
refer to them, use
%entity-name;Parameter entities use the
Percent symbol.. The definition also includes the %
between the ENTITY keyword and the name of the
entity.Defining parameter entities
]>]]>This may not seem particularly useful. It will be.For you to do…Add a general entity to
example.sgml.
]>
An example HTML file
This is a paragraph containing some text.
This paragraph contains some more text.
This paragraph might be right-justified.
The current version of this document is: &version;
]]>Validate the document using &man.nsgmls.1;Load example.sgml into your web browser
(you may need to copy it to example.html
before your browser recognises it as an HTML document).Unless your browser is very advanced, you won't see the entity
reference &version; replaced with the
version number. Most web browsers have very simplistic parsers
which do not handle proper SGMLThis is a shame. Imagine all the problems and hacks (such
as Server Side Includes) that could be avoided if they
did..The solution is to normalise your
document using an SGML normaliser. The normaliser reads in valid
SGML and outputs equally valid SGML which has been transformed in
some way. One of the ways in which the normaliser transforms the
SGML is to expand all the entity references in the document,
replacing the entities with the text that they represent.You can use &man.sgmlnorm.1; to do this.&prompt.user; sgmlnorm example.sgml > example.htmlYou should find a normalised (i.e., entity references
expanded) copy of your document in
example.html, ready to load into your web
browser.If you look at the output from &man.sgmlnorm.1; you will see
that it does not include a DOCTYPE declaration at the start. To
include this you need to use the
option;&prompt.user; sgmlnorm -d example.sgml > example.htmlUsing entities to include filesEntities (both general and parameter) are
particularly useful when used to include one file inside another.Using general entities to include filesSuppose you have some content for an SGML book organised into
files, one file per chapter, called
chapter1.sgml,
chapter2.sgml, and so forth, with a
book.sgml file that will contain these
chapters.In order to use the contents of these files as the values for your
entities, you declare them with the SYSTEM keyword.
This directs the SGML parser to use the contents of the named file as
the value of the entity.Using general entities to include files
]>
&chapter.1;
&chapter.2;
&chapter.3;
]]>When using general entities to include other files within a
document, the files being included
(chapter1.sgml,
chapter2.sgml, and so on) must
not start with a DOCTYPE declaration. This is a syntax
error.Using parameter entities to include filesRecall that parameter entities can only be used inside an SGML
context. Why then would you want to include a file within an SGML
context?You can use this to ensure that you can reuse your general
entities.Suppose that you had many chapters in your document, and you
reused these chapters in two different books, each book organising the
chapters in a different fashion.You could list the entities at the top of each book, but this
quickly becomes cumbersome to manage.Instead, place the general entity definitions inside one file,
and use a parameter entity to include that file within your
document.Using parameter entities to include filesFirst, place your entity definitions in a separate file, called
chapters.ent. This file contains the
following;
]]>Now create a parameter entity to refer to the contents of the
file. Then use the parameter entity to load the file into the
document, which will then make all the general entities available
for use. Then use the general entities as before;
%chapters;
]>
&chapter.1;
&chapter.2;
&chapter.3;
]]>For you to do…Use general entities to include filesCreate three files, para1.sgml,
para2.sgml, and
para3.sgml.Put content similar to the following in each file;This is the first paragraph.]]>Edit example.sgml so that it looks like
this;
]>
An example HTML file
The current version of this document is: &version;
¶1;
¶2;
¶3;
]]>Produce example.html by normalising
example.sgml.&prompt.user; sgmlnorm -d example.sgml > example.htmlLoad example.html in to your web
browser, and confirm that the
paran.sgml files
have been included in example.html.Use parameter entities to include filesYou must have taken the previous steps first.Edit example.sgml so that it looks like
this; %entities;
]>
An example HTML file
The current version of this document is: &version;
¶1;
¶2;
¶3;
]]>Create a new file, entities.sgml, with
this content:
]]>Produce example.html by normalising
example.sgml.&prompt.user; sgmlnorm -d example.sgml > example.htmlLoad example.html in to your web
browser, and confirm that the
paran.sgml files
have been included in example.html.Marked sectionsSGML provides a mechanism to indicate that particular pieces of the
document should be processed in a special way. These are termed
- “marked sections”.
+ marked sections.
Structure of a marked section<![ KEYWORD [
Contents of marked section
]]>As you would expect, being an SGML construct, a marked section
starts with <!.The first square bracket begins to delimit the marked
section.KEYWORD describes how this marked
section should be processed by the parser.The second square bracket indicates that the content of the marked
section starts here.The marked section is finished by closing the two square brackets,
and then returning to the document context from the SGML context with
>Marked section keywordsCDATA, RCDATAThese keywords denote the marked sections content
model, and allow you to change it from the
default.When an SGML parser is processing a document it keeps track
- of what is called the “content model”.
+ of what is called the content model.
Briefly, the content model describes what sort of content the
parser is expecting to see, and what it will do with it when it
finds it.The two content models you will probably find most useful are
CDATA and RCDATA.
- CDATA is for “Character Data”.
+ CDATA is for Character Data.
If the parser is in this content model then it is expecting to see
characters, and characters only. In this model the < and &
symbols lose their special status, and will be treated as ordinary
characters.
- RCDATA is for “Entity references and
- character data” If the parser is in this content model then it
+ RCDATA is for Entity references and
+ character data If the parser is in this content model then it
is expecting to see characters and entities.
< loses its special status, but & will still be treated as
starting the beginning of a general entity.This is particularly useful if you are including some verbatim
text that contains lots of < and & characters. While you
could go through the text ensuring that every < is converted to a
< and every & is converted to a &, it can be
easier to mark the section as only containing CDATA. When the SGML
parser encounters this it will ignore the < and & symbols
embedded in the content.Using a CDATA marked section<para>Here is an example of how you would include some text
that contained many < and & symbols. The sample
text is a fragment of HTML. The surrounding text (<para> and
<programlisting>) are from DocBook.</para>
<programlisting>
<![ CDATA [ This is a sample that shows you some of the elements within
HTML. Since the angle brackets are used so many times, it's
simpler to say the whole example is a CDATA marked section
than to use the entity names for the left and right angle
brackets throughout.
This is a listitem
This is a second listitem
This is a third listitem
This is the end of the example.
]]>
]]>
</programlisting>If you look at the source for this document you will see this
technique used throughout.INCLUDE and
IGNOREIf the keyword is INCLUDE then the contents
of the marked section will be processed. If the keyword is
IGNORE then the marked section is ignored and
will not be processed. It will not appear in the output.Using INCLUDE and
IGNORE in marked sections<![ INCLUDE [
This text will be processed and included.
]]>
<![ IGNORE [
This text will not be processed or included.
]]>By itself, this isn't too useful. If you wanted to remove text
from your document you could cut it out, or wrap it in
comments.It becomes more useful when you realise you can use parameter entities
to control this. Remember that parameter entities can only be used
in SGML contexts, and the keyword of a marked section
is an SGML context.For example, suppose that you produced a hard-copy version of
some documentation and an electronic version. In the electronic
version you wanted to include some extra content that wasn't to
appear in the hard-copy.Create a parameter entity, and set it's value to
INCLUDE. Write your document, using marked
sections to delimit content that should only appear in the
electronic version. In these marked sections use the parameter
entity in place of the keyword.When you want to produce the hard-copy version of the document,
change the parameter entity's value to IGNORE and
reprocess the document.Using a parameter entity to control a marked
section<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.0//EN" [
<!ENTITY % electronic.copy "INCLUDE">
]]>
...
<![ %electronic.copy [
This content should only appear in the electronic
version of the document.
]]>When producing the hard-copy version, change the entity's
definition to;<!ENTITY % electronic.copy "IGNORE">On reprocessing the document, the marked sections that use
%electronic.copy as their keyword will be
ignored.For you to do…Create a new file, section.sgml, that
contains the following;<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.0//EN" [
<!ENTITY % text.output "INCLUDE">
]>
<html>
<head>
<title>An example using marked sections</title>
</head>
<body>
<p>This paragraph <![ CDATA [contains many <
characters (< < < < <) so it is easier
to wrap it in a CDATA marked section ]]></p>
<![ IGNORE [
<p>This paragraph will definitely not be included in the
output.</p>
]]>
<![ [
<p>This paragraph might appear in the output, or it
might not.</p>
<p>Its appearance is controlled by the
parameter entity.</p>
]]>
</body>
</html>Normalise this file using &man.sgmlnorm.1; and examine the
output. Notice which paragraphs have appeared, which have
disappeared, and what has happened to the content of the CDATA
marked section.Change the definition of the text.output
entity from INCLUDE to
IGNORE. Re-normalise the file, and examine the
output to see what has changed. ConclusionThat is the conclusion of this SGML primer. For reasons of space
and complexity several things have not been covered in depth (or at
all). However, the previous sections cover enough SGML for you to be
able to follow the organisation of the FDP documentation.
diff --git a/en_US.ISO8859-1/books/fdp-primer/translations/chapter.sgml b/en_US.ISO8859-1/books/fdp-primer/translations/chapter.sgml
index cf8a38bcee..b5ebbeb5ad 100644
--- a/en_US.ISO8859-1/books/fdp-primer/translations/chapter.sgml
+++ b/en_US.ISO8859-1/books/fdp-primer/translations/chapter.sgml
@@ -1,482 +1,482 @@
TranslationsThis is the FAQ for people translating the FreeBSD documentation
(FAQ, Handbook, tutorials, man pages, and others) to different
languages.It is very heavily based on the translation FAQ
from the FreeBSD German Documentation Project, originally written by Frank
Gründer elwood@mc5sys.in-berlin.de and translated back to
English by Bernd Warken bwarken@mayn.de.The FAQ maintainer is Nik Clayton
nik@FreeBSD.org.Why a FAQ?More and more people are approaching the freebsd-doc mailing
list and volunteering to translate FreeBSD documentation to other
languages. This FAQ aims to answer their questions so they can start
translating documentation as quickly as possible.What do i18n and l10n
mean?i18n means
internationalisation and l10n
means localisation. They are just a convenient
shorthand.
- i18n can be read as “i” followed by
- 18 letters, followed by “n”. Similarly,
- l10n is “l” followed by 10 letters,
- followed by “n”.
+ i18n can be read as i followed by
+ 18 letters, followed by n. Similarly,
+ l10n is l followed by 10 letters,
+ followed by n.Is there a mailing list for translators?Yes, freebsd-translate@ngo.org.uk. Subscribe by
sending a message to
freebsd-translate-request@ngo.org.uk with the word
subscribe in the body of the message.You will receive a reply asking you to confirm your subscription
(in exactly the same manner as the FreeBSD lists at FreeBSD.org).The primary language of the mailing list is English. However,
posts in other languages will be accepted. The mailing list is not
moderated, but you need to be a member of the list before you can
post to it.The mailing list is archived, but they are not currently
searchable. Sending the message help to
majordomo@ngo.org.uk will send back instructions on
how to access the archive.It is expected that the mailing list will transfer to FreeBSD.org and therefore become
official in the near future.Are more translators needed?Yes. The more people work on translation the faster it gets
done, and the faster changes to the English documentation are
mirrored in the translated documents.You do not have to be a professional translator to be able to
help.What languages do I need to know?Ideally, you will have a good knowledge of written English, and
obviously you will need to be fluent in the language you are
translating to.English is not strictly necessary. For example, you could do a
Hungarian translation of the FAQ from the Spanish
translation.What software do I need to know?It is strongly recommended that you maintain a local copy of the
FreeBSD CVS repository (at least the documentation part) either
using CTM or
CVSup. The "Staying current with FreeBSD"
chapter in the Handbook explains how to use these
applications.You should be comfortable using CVS.
This will allow you to see what has changed between different
versions of the files that make up the documentation.[XXX To Do -- write a tutorial that shows how to use CVSup to
get just the documentation, check it out, and see what's changed
between two arbitrary revisions]How do I find out who else might be translating to the same
language?The Documentation
Project translations page lists the translation efforts
that are currently known about. If others are already working
on translating documentation to your language, please don't
duplicate their efforts. Instead, contact them to see how you can
help.If no one is listed on that page as translating for your
language, then send a message to
freebsd-doc@FreeBSD.org in case someone else is
thinking of doing a translation, but hasn't announced it yet.No one else is translating to my language. What do I do?
- Congratulations, you have just started the “FreeBSD
+ Congratulations, you have just started the FreeBSD
your-language-here Documentation
- Translation Project”. Welcome aboard.
+ Translation Project. Welcome aboard.First, decide whether or not you've got the time to spare. Since
you are the only person working on your language at the moment it is
going to be your responsibility to publicise your work and
coordinate any volunteers that might want to help you.Write an e-mail to the Documentation Project mailing list,
announcing that you are going to translate the documentation, so the
Documentation Project translations page can be maintained.You should subscribe to the
freebsd-translate@ngo.org.uk mailing list (as
described earlier).If there is already someone in your country providing FreeBSD
mirroring services you should contact them and ask if you can
have some webspace for your project, and possibly an e-mail
address or mailing list services.Then pick a document and start translating. It is best to start
with something fairly small—either the FAQ, or one of the
tutorials.I've translated some documentation, where do I send it?That depends. If you are already working with a translation team
(such as the Japanese team, or the German team) then they will have
their own procedures for handling submitted documentation, and these
will be outlined on their web pages.If you are the only person working on a particular language (or
you are responsible for a translation project and want to submit
your changes back to the FreeBSD project) then you should send your
translation to the FreeBSD project (see the next question).I'm the only person working on translating to this language, how
do I submit my translation?orWe're a translation team, and want to submit documentation that
our members have translated for us?First, make sure your translation is organised properly. This
means that it should drop in to the existing documentation tree and
build straight away.Currently, the FreeBSD documentation is stored in a top level
directory called doc/. Directories below this
are named according to the language code they are written in, as
defined in ISO639 (/usr/share/misc/iso639 on a
version of FreeBSD newer than 20th January 1999).If your language can be encoded in different ways (for example,
Chinese) then there should be directories below this, one for each
encoding format you have provided.Finally, you should have directories for each document.For example, a hypothetical Swedish translation might look
likedoc/
sv_SE.ISO8859-1/
Makefile
books/
faq/
Makefile
book.sgmlsv_SE.ISO8859-1 is the name of the
translation, in
lang.encoding
form. Note the
two Makefiles, which will be used to build the documentation.Use &man.tar.1; and &man.gzip.1; to compress up your
documentation, and send it to the project.&prompt.user; cd doc
&prompt.user; tar cf swedish-docs.tar sv
&prompt.user; gzip -9 swedish-docs.tarPut swedish-docs.tar.gz somewhere. If you
do not have access to your own webspace (perhaps your ISP does not
let you have any) then you can e-mail Nik Clayton
nik@FreeBSD.org, and arrange to e-mail the files
when it is convenient.Either way, you should use &man.send-pr.1; to submit a report
indicating that you have submitted the documentation. It would be
very helpful if you could get other people to look over your
translation and double check it first, since it is unlikely that the
person committing it will be fluent in the language.Someone (probably the Documentation Project Manager, currently
Nik Clayton nik@FreeBSD.org) will then take your
translation and confirm that it builds. In particular, the
following things will be looked at:Do all your files use RCS strings (such as "ID")?Does make all in the
sv_SE.ISO8859-1 directory work correctly?Does make install work correctly?If there are any problems then whoever is looking at the
submission will get back to you to try and work them out.If there are no problems your translation will be committed
as soon as possible.Can I include language or country specific text in my
translation?We would prefer that you did not.For example, suppose that you are translating the Handbook to
Korean, and want to include a section about retailers in Korea in
your Handbook.There's no real reason why that information should not be in the
English (or German, or Spanish, or Japanese, or …) versions
as well. It is feasible that an English speaker in Korea might try
and pick up a copy of FreeBSD whilst over there. It also helps
increase FreeBSD's perceived presence around the globe, which is not
a bad thing.If you have country specific information, please submit it as a
change to the English Handbook (using &man.send-pr.1;) and then
translate the change back to your language in the translated
Handbook.Thanks.How should language specific characters be included?Non-ASCII characters in the documentation should be included
using SGML entities.Briefly, these look like an ampersand (&), the name of the
entity, and a semi-colon (;).The entity names are defined in ISO8879, which is in the ports
tree as textproc/iso8879.A few examples includeEntityAppearanceDescriptionéé
- Small “e” with an acute accent
+ Small e with an acute accentÉÉ
- Large “E” with an acute accent
+ Large E with an acute accentüü
- Small “u” with an umlaut
+ Small u with an umlautAfter you have installed the iso8879 port, the files in
/usr/local/share/sgml/iso8879 contain the
complete list.Addressing the readerIn the English documents, the reader is addressed as
- “you”, there is no formal/informal distinction as there
+ you, there is no formal/informal distinction as there
is in some languages.If you are translating to a language which does distinguish, use
whichever form is typically used in other technical documentation in
your language. If in doubt, use a mildly polite form.Do I need to include any additional information in my
translations?Yes.The header of the English version of each document will look
something like this;<!--
The FreeBSD Documentation Project
$FreeBSD: doc/en_US.ISO8859-1/books/fdp-primer/translations/chapter.sgml,v 1.5 2000/07/07 18:38:38 dannyboy Exp $
-->The exact boilerplate may change, but it will always include a
$FreeBSD$ line and the phrase The FreeBSD Documentation
Project.
Note that the $FreeBSD part is expanded automatically by
CVS, so it should be empty (just
$FreeBSD$) for new files.Your translated documents should include their own
$FreeBSD$ line, and change the
FreeBSD Documentation Project line to
The FreeBSD language
Documentation Project.In addition, you should add a third line which indicates which
revision of the English text this is based on.So, the Spanish version of this file might start<!--
The FreeBSD Spanish Documentation Project
$FreeBSD: doc/es_ES.ISO8859-1/books/fdp-primer/translations/chapter.sgml,v 1.3 1999/06/24 19:12:32 jesusr Exp $
Original revision: 1.11
-->
diff --git a/en_US.ISO8859-1/books/fdp-primer/writing-style/chapter.sgml b/en_US.ISO8859-1/books/fdp-primer/writing-style/chapter.sgml
index e382f0ae26..18c1e0ecdc 100644
--- a/en_US.ISO8859-1/books/fdp-primer/writing-style/chapter.sgml
+++ b/en_US.ISO8859-1/books/fdp-primer/writing-style/chapter.sgml
@@ -1,388 +1,388 @@
Writing styleIn order to promote consistency between the myriad authors of the
FreeBSD documentation, some guidelines have been drawn up for authors to
follow.Do not use contractionsDo not use contractions. Always spell the phrase out in full.
- “Don't use contractions” would be wrong.
+ Don't use contractions would be wrong.
Avoiding contractions makes for a more formal tone, is more
precise, and is slightly easier for translators.Use the serial commaIn a list of items within a paragraph, separate each item from
the others with a comma. Seperate the last item from the others with
- a comma and the word “and”.
+ a comma and the word and.
For example, look at the following:
This is a list of one, two and three items.
- Is this a list of three items, “one”,
- “two”, and “three”, or a list of two items,
- “one” and “two and three”?
+ Is this a list of three items, one,
+ two, and three, or a list of two items,
+ one and two and three?It is better to be explicit and include a serial comma:
This is a list of one, two, and three items.
Avoid redundant phrases
- Try not to use redundant phrases. In particular, “the
- command”, “the file”, and “man
- command” are probably redundant.
+ Try not to use redundant phrases. In particular, the
+ command, the file, and man
+ command are probably redundant.These two examples show this for commands. The second example
is preferred.Use the command cvsup to update your
sourcesUse cvsup to update your sourcesThese two examples show this for filenames. The second example
is preferred.… in the filename
/etc/rc.local…… in
/etc/rc.local…These two examples show this for manual references. The second
example is preferred (the second example uses
citerefentry).See man csh for more
information.See &man.csh.1;Two spaces at the end of sentencesAlways use two spaces at the end of sentences, as this
improves readability, and eases use of tools such as
emacs.While it may be argued that a capital letter following
a period denotes a new sentence, this is not the case, especially
in name usage. Jordan K. Hubbard is a good
example; it has a capital H following a
period and a space, and there certainly isn't a new sentence
there.For more information about writing style, see Elements of
Style, by William Strunk.Style guideTo keep the source for the Handbook consistent when many different
people are editing it, please follow these style conventions.Letter caseTags are entered in lower case, <para>,
not<PARA>.Text that appears in SGML contexts is generally written in upper
case, <!ENTITY…>, and
<!DOCTYPE…>, not<!entity…> and
<!doctype…>.IndentationEach file starts with indentation set at column 0,
regardless of the indentation level of the file
which might contain this one.Every start tag increases the indentation level by 2 spaces, and
every end tag decreases the indentation level by 2 spaces. Replace
as many leading spaces with tabs as appropriate. Do not use
spaces in front of tabs, and do not add extraneous whitespace at the
end of a line. Content
within elements should be indented by two spaces if the content runs
over more than one line.For example, the source for this section looks something
like:......IndentationEach file starts with indentation set at column 0,
regardless of the indentation level of the file
which might contain this one.Every start tag increases the indentation level by 2 spaces, and
every end tag decreases the indentation level by 2 spaces. Content
within elements should be indented by two spaces if the content runs
over more than one line.
...
]]>
If you use Emacs or
Xemacs to edit the files then
sgml-mode should be loaded automatically, and the
Emacs local variables at the bottom of each file should enforce these
styles.Tag styleTag spacingTags that start at the same indent as a previous tag
should be separated by a blank line, and those that are not
at the same indent as a previous tag should not:NISOctober 1999
...
...
...............
]]>Separating tagsTags like itemizedlist which will
always have further tags inside them, and in fact don't take
character data themselves, are always on a line by
themselves.Tags like para and
term don't need other tags to contain
normal character data, and their contents begin immediately
after the tag, on the same line.The same applies to when these two types of tags
close.This leads to an obvious problem when mixing these
tags.When a starting tag which cannot contain character data
directly follows a tag of the type that requires other tags
within it to use character data, they are on separate lines.
The second tag should be properly indented.When a tag which can contain character data closes
directly after a tag which cannot contain character data
closes, they co-exist on the same line.White space changesWhen committing changes, do not commit changes to the
content at the same time as changes to the
formatting.This is so that the teams that convert the Handbook to other
languages can quickly see what content has actually changed in your
commit, without having to decide whether a line has changed because of
the content, or just because it has been refilled.For example, if you have added two sentences to a paragraph, such
that the line lengths on the paragraph now go over 80 columns, first
commit your change with the too-long line lengths. Then fix the line
wrapping, and commit this second change. In the commit message for
the second change, be sure to indicate that this is a whitespace-only
change, and that the translation team can ignore it.Word listThe following is a small list of words spelled the way they
should be used in the FreeBSD Documentation Project. If the
word you are looking for is not in this list, then please
consult the O'Reilly
word list.2.2.X4.X-STABLEDoS (Denial of Service)FreeBSD Ports CollectionInternetCDROMMHzports collectionUnixemailmanual page(s)mail servername serverweb server
diff --git a/en_US.ISO8859-1/books/handbook/advanced-networking/chapter.sgml b/en_US.ISO8859-1/books/handbook/advanced-networking/chapter.sgml
index c91f780a66..5b3b2b5e43 100644
--- a/en_US.ISO8859-1/books/handbook/advanced-networking/chapter.sgml
+++ b/en_US.ISO8859-1/books/handbook/advanced-networking/chapter.sgml
@@ -1,4987 +1,4987 @@
Advanced NetworkingSynopsisThis chapter will cover some of the more frequently used network
services on Unix systems. We will cover how to define, setup, test and
maintain all of the network services that FreeBSD utilizes. In addition,
there have been example configuration files included throughout this
chapter for you to benefit from.After reading this chapter, you will know:The basics of gateways and routes.How to make FreeBSD act as a bridge.How to setup a network file system.How to setup network booting on a diskless machine.How to setup a network information server for sharing user
accounts.How to setup automatic network settings using DHCP.How to setup a domain name server.How to setup network address translation.How to manage the inetd daemon.Before reading this chapter, you should:understand the basics of the /etc/rc scripts.be familiar with basic network terminology.CoranthGryphonContributed by Gateways and RoutesroutinggatewaysubnetFor one machine to be able to find another over a network, there
must be a
mechanism in place to describe how to get from one to the other. This is
called routing. A route is a defined pair of addresses: a
destination and a gateway. The pair
indicates that if you are trying to get to this
destination, communicate through this
gateway. There are three types of destinations:
individual hosts, subnets, and default. The
default route is used if none of the other routes apply.
We will talk a little bit more about default routes later on. There are
also three types of gateways: individual hosts, interfaces (also called
links), and Ethernet hardware addresses (MAC addresses).
An ExampleTo illustrate different aspects of routing, we will use the
following example from netstat:&prompt.user; netstat -r
Routing tables
Destination Gateway Flags Refs Use Netif Expire
default outside-gw UGSc 37 418 ppp0
localhost localhost UH 0 181 lo0
test0 0:e0:b5:36:cf:4f UHLW 5 63288 ed0 77
10.20.30.255 link#1 UHLW 1 2421
foobar.com link#1 UC 0 0
host1 0:e0:a8:37:8:1e UHLW 3 4601 lo0
host2 0:e0:a8:37:8:1e UHLW 0 5 lo0 =>
host2.foobar.com link#1 UC 0 0
224 link#1 UC 0 0default routeThe first two lines specify the default route (which we
will cover in the next
section) and the localhost route.loopback deviceThe interface (Netif column) that it specifies
to use for localhost is
lo0, also known as the loopback device. This
says to keep all traffic for this destination internal, rather than
sending it out over the LAN, since it will only end up back where it
started.EthernetMAC addressThe next thing that stands out are the addresses beginning
with 0:e0:. These are Ethernet
hardware addresses, which are also known as MAC addresses.
FreeBSD will automatically identify any hosts
(test0 in the example) on the local Ethernet and add
a route for that host, directly to it over the Ethernet interface,
ed0. There is also a timeout
(Expire column) associated with this type of route,
which is used if we fail to hear from the host in a specific amount of
time. In this case the route will be automatically deleted. These
hosts are identified using a mechanism known as RIP (Routing
Information Protocol), which figures out routes to local hosts based
upon a shortest path determination.subnetFreeBSD will also add subnet routes for the local subnet (10.20.30.255 is the broadcast address for the
subnet 10.20.30, and foobar.com is the domain name associated
with that subnet). The designation link#1 refers
to the first Ethernet card in the machine. You will notice no
additional interface is specified for those.Both of these groups (local network hosts and local subnets) have
their routes automatically configured by a daemon called
routed. If this is not run, then only
routes which are statically defined (ie. entered explicitly) will
exist.The host1 line refers to our host, which it
knows by Ethernet address. Since we are the sending host, FreeBSD
knows to use the loopback interface (lo0)
rather than sending it out over the Ethernet interface.The two host2 lines are an example of what
happens when we use an &man.ifconfig.8; alias (see the section of Ethernet for
reasons why we would do this). The => symbol
after the lo0 interface says that not only
are we using the loopback (since this is address also refers to the
local host), but specifically it is an alias. Such routes only show
up on the host that supports the alias; all other hosts on the local
network will simply have a link#1 line for
such.The final line (destination subnet 224) deals
with MultiCasting, which will be covered in a another section.The other column that we should talk about are the
Flags. Each route has different attributes that
are described in the column. Below is a short table of some of these
flags and their meanings:UUp: The route is active.HHost: The route destination is a single host.GGateway: Send anything for this destination on to this
remote system, which will figure out from there where to send
it.SStatic: This route was configured manually, not
automatically generated by the system.CClone: Generates a new route based upon this route for
machines we connect to. This type of route is normally used
for local networks.WWasCloned: Indicated a route that was auto-configured
based upon a local area network (Clone) route.LLink: Route involves references to Ethernet
hardware.Default Routesdefault routeWhen the local system needs to make a connection to remote host,
it checks the routing table to determine if a known path exists. If
the remote host falls into a subnet that we know how to reach (Cloned
routes), then the system checks to see if it can connect along that
interface.If all known paths fail, the system has one last option: the
default route. This route is a special type of gateway
route (usually the only one present in the system), and is always
marked with a c in the flags field. For hosts on a
local area network, this gateway is set to whatever machine has a
direct connection to the outside world (whether via PPP link,
DSL, cable modem, T1, or another network interface).If you are configuring the default route for a machine which
itself is functioning as the gateway to the outside world, then the
default route will be the gateway machine at your Internet Service
Provider's (ISP) site.Let us look at an example of default routes. This is a common
configuration:
[Local2] <--ether--> [Local1] <--PPP--> [ISP-Serv] <--ether--> [T1-GW]
The hosts Local1 and
Local2 are at your site.
Local1 is connected to an ISP via a dial up
PPP connection. This PPP server computer is connected through
a local area network to another gateway computer with an
external interface to the ISPs Internet feed.The default routes for each of your machines will be:HostDefault GatewayInterfaceLocal2Local1EthernetLocal1T1-GWPPPA common question is Why (or how) would we set the T1-GW to
be the default gateway for Local1, rather than the ISP server it is
connected to?.Remember, since the PPP interface is using an address on the ISP's
local network for your side of the connection, routes for any other
machines on the ISP's local network will be automatically generated.
Hence, you will already know how to reach the T1-GW machine, so there
is no need for the intermediate step of sending traffic to the ISP
server.As a final note, it is common to use the address X.X.X.1 as the gateway address for your local
network. So (using the same example), if your local class-C address
space was 10.20.30 and your ISP was
using 10.9.9 then the default routes
would be:HostDefault RouteLocal2 (10.20.3.2)Local1 (10.20.30.1)Local1 (10.20.30.1, 10.9.9.30)T1-GW (10.9.9.1)Dual Homed Hostsdual homed hostsThere is one other type of configuration that we should cover, and
that is a host that sits on two different networks. Technically, any
machine functioning as a gateway (in the example above, using a PPP
connection) counts as a dual-homed host. But the term is really only
used to refer to a machine that sits on two local-area
networks.In one case, the machine has two Ethernet cards, each having an
address on the separate subnets. Alternately, the machine may only
have one Ethernet card, and be using &man.ifconfig.8; aliasing. The former is
used if two physically separate Ethernet networks are in use, the
latter if there is one physical network segment, but two logically
separate subnets.Either way, routing tables are set up so that each subnet knows
that this machine is the defined gateway (inbound route) to the other
subnet. This configuration, with the machine acting as a Bridge
between the two subnets, is often used when we need to implement
packet filtering or firewall security in either or both
directions.Routing Propagationrouting propagationWe have already talked about how we define our routes to the
outside world, but not about how the outside world finds us.We already know that routing tables can be set up so that all
traffic for a particular address space (in our examples, a class-C
subnet) can be sent to a particular host on that network, which will
forward the packets inbound.When you get an address space assigned to your site, your service
provider will set up their routing tables so that all traffic for your
subnet will be sent down your PPP link to your site. But how do sites
across the country know to send to your ISP?There is a system (much like the distributed DNS information) that
keeps track of all assigned address-spaces, and defines their point of
connection to the Internet Backbone. The Backbone are
the main trunk lines that carry Internet traffic across the country,
and around the world. Each backbone machine has a copy of a master
set of tables, which direct traffic for a particular network to a
specific backbone carrier, and from there down the chain of service
providers until it reaches your network.It is the task of your service provider to advertise to the
backbone sites that they are the point of connection (and thus the
path inward) for your site. This is known as route
propagation.TroubleshootingtracerouteSometimes, there is a problem with routing propagation, and some
sites are unable to connect to you. Perhaps the most useful command
for trying to figure out where a routing is breaking down is the
&man.traceroute.8; command. It is equally useful if you cannot seem
to make a connection to a remote machine (i.e. &man.ping.8;
fails).The &man.traceroute.8; command is run with the name of the remote
host you are trying to connect to. It will show the gateway hosts
along the path of the attempt, eventually either reaching the target
host, or terminating because of a lack of connection.For more information, see the manual page for
&man.traceroute.8;.StevePetersonWritten by BridgingIntroductionIP subnetbridgeIt is sometimes useful to divide one physical network (such as an
Ethernet segment) into two separate network segments without having
to create IP subnets and use a router to connect the segments
together. A device that connects two networks together in this
fashion is called a bridge. A FreeBSD system with two network
interface cards can act as a bridge.The bridge works by learning the MAC layer addresses
(Ethernet addresses) of the devices on each of its network interfaces.
It forwards traffic between two networks only when its source and
destination are on different networks.In many respects, a bridge is like an Ethernet switch with very
few ports.Situations Where Bridging Is AppropriateThere are two common situations in which a bridge is used
today.High Traffic on a SegmentSituation one is where your physical network segment is
overloaded with traffic, but you do not want for whatever reason to
subnet the network and interconnect the subnets with a
router.Let us consider an example of a newspaper where the Editorial and
Production departments are on the same subnetwork. The Editorial
users all use server A for file service, and the Production users
are on server B. An Ethernet is used to connect all users together,
and high loads on the network are slowing things down.If the Editorial users could be segregated on one network
segment and the Production users on another, the two network
segments could be connected with a bridge. Only the network traffic
destined for interfaces on the "other" side of the bridge would be
sent to the other network, reducing congestion on each network
segment.Filtering/Traffic Shaping FirewallfirewallIP MasqueradingThe second common situation is where firewall functionality is
needed without IP Masquerading (NAT).An example is a small company that is connected via DSL or ISDN
to their ISP. They have a 13 globally-accessible IP addresses
from their ISP and have 10 PCs on their network. In this situation, using a
router-based firewall is difficult because of subnetting
issues.routerDSLISDNA bridge-based firewall can be configured and dropped into the
path just downstream of their DSL/ISDN router without any IP
numbering issues.Configuring a BridgeNetwork Interface Card SelectionA bridge requires at least two network cards to function.
Unfortunately, not all network interface cards as of FreeBSD 4.0
support bridging. Read &man.bridge.4; for details on the cards that
are supported.Install and test the two network cards before continuing.Kernel Configuration Changeskernel configurationkernel configurationoptions BRIDGETo enable kernel support for bridging, add the:options BRIDGEstatement to your kernel configuration file, and rebuild your
kernel.Firewall SupportfirewallIf you are planning to use the bridge as a firewall, you will
need to add the IPFIREWALL option as well. Read for general information on configuring the
bridge as a firewall.If you need to allow non-IP packets (such as ARP) to flow
through the bridge, there is an undocumented firewall option that
must be set. This option is
IPFIREWALL_DEFAULT_TO_ACCEPT. Note that this
changes the default rule for the firewall to accept any packet.
Make sure you know how this changes the meaning of your ruleset
before you set it.Traffic Shaping SupportIf you want to use the bridge as a traffic shaper, you will need
to add the DUMMYNET option to your kernel
configuration. Read &man.dummynet.4; for further
information.Enabling the BridgeAdd the line:net.link.ether.bridge=1to /etc/sysctl.conf to enable the bridge at
runtime. If you want the bridged packets to be filtered by &man.ipfw.8;,
you should also add:net.link.ether.bridge_ipfw=1as well.PerformanceMy bridge/firewall is a Pentium 90 with one 3Com 3C900B and one
3C905B. The protected side of the network runs at 10mbps half duplex
and the connection between the bridge and my router (a Cisco 675) runs
at 100mbps full duplex. With no filtering enabled, I have found that
the bridge adds about 0.4 milliseconds of latency to pings from the
protected 10mbps network to the Cisco 675.Other InformationIf you want to be able to telnet into the bridge from the network,
it is OK to assign one of the network cards an IP address. The
consensus is that assigning both cards an address is a bad
idea.If you have multiple bridges on your network, there cannot be more
than one path between any two workstations. Technically, this means
that there is no support for spanning tree link management.BillSwingleWritten by NFSNFSAmong the many different file systems that FreeBSD supports is
the Network File System or NFS. NFS allows you
to share directories and files on one machine with others
via the network they are attached to. Using NFS, users and
programs can access files on remote systems as if they were local
files.NFS has several benefits:Local workstations do not need as much disk space because
commonly used data can be stored on a single machine and still
remain accessible to everyone on the network.There is no need for users to have unique home directories
on every machine on your network. Once they have an established
directory that is available via NFS it can be accessed from
anywhere.Storage devices such as floppies and CDROM drives can be
used by other machines on the network eliminating the need for
extra hardware.How It WorksNFS is composed of two sides – a client side and a
server side. Think of it as a want/have relationship. The client
wants the data that the server side
has. The server shares its data with the
client. In order for this system to function properly a few
processes have to be configured and running.The server has to be running the following daemons:NFSserverportmapmountdnfsdDaemonDescriptionnfsdThe NFS Daemon which services requests from NFS
clients.mountdThe NFS Mount Daemon which actually carries out
requests that &man.nfsd.8; passes on to it.portmap The portmapper daemon which
allows NFS clients to find out which port the NFS server
is using.The client side only needs to run a single daemon:NFSclientnfsiodnfsiodThe NFS async I/O Daemon which services requests
from its NFS server.Configuring NFSNFSconfigurationLuckily for us, on a FreeBSD system this setup is a snap. The
processes that need to be running can all be run at boot time with
a few modifications to your /etc/rc.conf
file.On the NFS server make sure you have:portmap_enable="YES"
nfs_server_enable="YES"
nfs_server_flags="-u -t -n 4"
mountd_flags="-r"mountd is automatically run whenever the
NFS server is enabled. The and
flags to nfsd tell it to
serve UDP and TCP clients. The flag tells
nfsd to start 4 copies of itself.On the client, make sure you have:nfs_client_enable="YES"
nfs_client_flags="-n 4"Like nfsd, the tells
nfsiod to start 4 copies of itself.The last configuration step requires that you create a file
called /etc/exports. The exports file
specifies which file systems on your server will be shared
(a.k.a., exported) and with what clients they will
be shared. Each line in the file specifies a file system to be
shared. There are a handful of options that can be used in this
file but only a few will be mentioned here. You can find out
about the rest in the &man.exports.5; manual page.Here are a few example /etc/exports
entries:NFSexporting filesystemsThe following line exports /cdrom to
three silly machines that have the same domain name as the server
(hence the lack of a domain name for each) or have entries in your
/etc/hosts file. The
flag makes the shared file system read-only. With this flag, the
remote system will not be able to make any changes to the
shared file system./cdrom -ro moe larry curlyThe following line exports /home to three
hosts by IP address. This is a useful setup if you have a
private network but do not have DNS running. The
flag allows all the directories below
the specified file system to be exported as well./home -alldirs 10.0.0.2 10.0.0.3 10.0.0.4The following line exports /a to two
machines that have different domain names than the server. The
flag allows
the root user on the remote system to write to the shared
file system as root. Without the -maproot=0 flag even if
someone has root access on the remote system they will not
be able to modify files on the shared file system./a -maproot=0 host.domain.com box.example.comIn order for a client to access- an exported file system it must
have permission to do so. Make sure your client is listed in your
/etc/exports file.It is important to remember that you must restart mountd
whenever you modify /etc/exports so that
your changes take effect. This can be accomplished by sending
the hangup signal to the mountd process :&prompt.root; kill -HUP `cat /var/run/mountd.pid`Now that you have made all these changes you can just reboot
and let FreeBSD start everything for you at boot time or you can
run the following commands as root:On the NFS server:&prompt.root; portmap
&prompt.root; nfsd -u -t -n 4
&prompt.root; mountd -rOn the NFS client:&prompt.root; nfsiod -n 4Now you should be ready to actually mount a remote file
system. This can be done one of two ways. In these examples the
server's name will be server and the client's
name will be client. If you just want to
temporarily mount a remote file system or just want to test out
your configuration you can run a command like this as root on the
client:NFSmounting filesystems&prompt.root; mount server:/home /mntThis will mount the /home directory on the
server at
/mnt on the client. If everything is setup
correctly you should be able to go into /mnt on the client and see
all the files that are on the server.If you want to permanently (each time you reboot) mount a
remote file system you need to add it to your
/etc/fstab file. Here is an example
line:server:/home /mnt nfs rw 0 0Read the &man.fstab.5; manual page for more options.Practical UsesThere are many very cool uses for NFS. Some of the more common
ones are listed below.NFSusesHave several machines on a network and share a CDROM or
floppy drive among them. This is cheaper and often more
convenient.With so many machines on a network, it gets old having your
personal files strewn all over the place. You can have a
central NFS server that houses all user home directories and
shares them with the rest of the machines on the LAN, so no
matter where you log in you will have the same home
directory.When you get to reinstalling FreeBSD on one of your
machines, NFS is the way to go! Just pop your distribution
CDROM into your file server and away you go!Have a common /usr/ports/distfiles
directory that all your machines share. That way, when you go
to install a port that you have already installed on a different
machine, you do not have to download the source all over
again!WylieStilwellContributed by ChernLeeRewritten by amdamdautomatic mounter daemon&man.amd.8;, which is also known as the automatic mounter
daemon, is a useful utility used for automatically mounting a
remote filesystem whenever a file or directory within that
filesystem is accessed. Filesystems that are inactive for a
period of time will also be automatically unmounted by
amd. Using
amd provides a simplistic alternative
to static mounts.amd operates by attaching
itself as an NFS server to the /host and
/net directories. When a file is accessed
within one of these directories, amd
looks up the corresponding remote mount and automatically mounts
it. /net is used to mount an exported
filesystem from an IP address, while /host
is used to mount an export from a remote hostname.An access to a file within
/host/foobar/usr would tell
amd to attempt to mount the
/usr export on the host
foobar.Mounting an Export with amd&prompt.user; showmount -e foobar
Exports list on foobar:
/usr 10.10.10.0
/a 10.10.10.0
&prompt.user; cd /host/foobar/usrAs seen in the example, the showmount shows
/usr as an export. When changing directories to
/host/foobar/usr, amd
attempts to resolve the hostname foobar and
automatically mount the desired export.amd can be started through the
rc.conf system by placing the following lines in
/etc/rc.conf:amd_enable="YES"Additionally, custom flags can be passed to
amd from the
amd_flags option. By default,
amd_flags is set to:amd_flags="-a /.amd_mnt -l syslog /host /etc/amd.map /net /etc/amd.map"The /etc/amd.map file defines the
default options that exports are mounted with. The
/etc/amd.conf file defines some of the more
advanced features of amd.Consult the &man.amd.8; and &man.amd.conf.5; man pages for more
information.JohnLindContributed by Problems Integrating with Other SystemsCertain Ethernet adapters for ISA PC systems have limitations
which can lead to serious network problems, particularly with NFS.
This difficulty is not specific to FreeBSD, but FreeBSD systems
are affected by it.The problem nearly always occurs when (FreeBSD) PC systems are
networked with high-performance workstations, such as those made
by Silicon Graphics, Inc., and Sun Microsystems, Inc. The NFS
mount will work fine, and some operations may succeed, but
suddenly the server will seem to become unresponsive to the
client, even though requests to and from other systems continue to
be processed. This happens to the client system, whether the
client is the FreeBSD system or the workstation. On many systems,
there is no way to shut down the client gracefully once this
problem has manifested itself. The only solution is often to
reset the client, because the NFS situation cannot be
resolved.Though the correct solution is to get a higher
performance and capacity Ethernet adapter for the FreeBSD system,
there is a simple workaround that will allow satisfactory
operation. If the FreeBSD system is the
server, include the option
on the mount from the client. If the
FreeBSD system is the client, then mount the
NFS file system with the option . These
options may be specified using the fourth field of the
fstab entry on the client for automatic
mounts, or by using the parameter of the mount
command for manual mounts.It should be noted that there is a different problem,
sometimes mistaken for this one, when the NFS servers and clients
are on different networks. If that is the case, make
certain that your routers are routing the
necessary UDP information, or you will not get anywhere, no matter
what else you are doing.In the following examples, fastws is the host
(interface) name of a high-performance workstation, and
freebox is the host (interface) name of a FreeBSD
system with a lower-performance Ethernet adapter. Also,
/sharedfs will be the exported NFS
filesystem (see &man.exports.5;), and
/project will be the mount point on the
client for the exported file system. In all cases, note that
additional options, such as or
and may be desirable in
your application.Examples for the FreeBSD system (freebox) as
the client: in /etc/fstab on freebox:fastws:/sharedfs /project nfs rw,-r=1024 0 0As a manual mount command on freebox:&prompt.root; mount -t nfs -o -r=1024 fastws:/sharedfs /projectExamples for the FreeBSD system as the server: in
/etc/fstab on fastws:freebox:/sharedfs /project nfs rw,-w=1024 0 0As a manual mount command on fastws:&prompt.root; mount -t nfs -o -w=1024 freebox:/sharedfs /projectNearly any 16-bit Ethernet adapter will allow operation
without the above restrictions on the read or write size.For anyone who cares, here is what happens when the failure
occurs, which also explains why it is unrecoverable. NFS
typically works with a block size of 8k (though it
may do fragments of smaller sizes). Since the maximum Ethernet
packet is around 1500 bytes, the NFS block gets
split into multiple Ethernet packets, even though it is still a
single unit to the upper-level code, and must be received,
assembled, and acknowledged as a unit. The
high-performance workstations can pump out the packets which
comprise the NFS unit one right after the other, just as close
together as the standard allows. On the smaller, lower capacity
cards, the later packets overrun the earlier packets of the same
unit before they can be transferred to the host and the unit as a
whole cannot be reconstructed or acknowledged. As a result, the
workstation will time out and try again, but it will try again
with the entire 8K unit, and the process will be repeated, ad
infinitum.By keeping the unit size below the Ethernet packet size
limitation, we ensure that any complete Ethernet packet received
can be acknowledged individually, avoiding the deadlock
situation.Overruns may still occur when a high-performance workstations
is slamming data out to a PC system, but with the better cards,
such overruns are not guaranteed on NFS units. When
an overrun occurs, the units affected will be retransmitted, and
there will be a fair chance that they will be received, assembled,
and acknowledged.MartinRentersContributed by Diskless Operationdiskless workstationnetboot.com/netboot.rom
allow you to boot your FreeBSD machine over the network and run FreeBSD
without having a disk on your client. Under 2.0 it is now possible to
have local swap. Swapping over NFS is also still supported.Supported Ethernet cards include: Western Digital/SMC 8003, 8013,
8216 and compatibles; NE1000/NE2000 and compatibles (requires
recompile)Setup InstructionsFind a machine that will be your server. This machine will
require enough disk space to hold the FreeBSD 2.0 binaries and
have bootp, tftp and NFS services available. Tested
machines:HP-UXHP9000/8xx running HP-UX 9.04 or later (pre 9.04 does not
work)SolarisSun/Solaris 2.3. (you may need to get bootp)Set up a bootp server to provide the client with IP address, gateway,
netmask.diskless:\
:ht=ether:\
:ha=0000c01f848a:\
:sm=255.255.255.0:\
:hn:\
:ds=192.1.2.3:\
:ip=192.1.2.4:\
:gw=192.1.2.5:\
:vm=rfc1048:TFTPbootpSet up a TFTP server (on same machine as bootp server) to
provide booting information to client. The name of this file is
cfg.X.X.X.X (or
/tftpboot/cfg.X.X.X.X,
it will try both) where X.X.X.X is the
IP address of the client. The contents of this file can be any
valid netboot commands. Under 2.0, netboot has the following
commands:helpprint help listip
print/set client's IP addressserver
print/set bootp/tftp server addressnetmask
print/set netmaskhostname nameprint/set hostnamekernel
print/set kernel namerootfs
print/set root filesystemswapfs
print/set swap filesystemswapsize
set diskless swapsize in KBytesdiskbootboot from diskautobootcontinue boot processtrans
|turn transceiver on|offflags
set boot flagsA typical completely diskless config file might contain:rootfs 192.1.2.3:/rootfs/myclient
swapfs 192.1.2.3:/swapfs
swapsize 20000
hostname myclient.mydomainA config file for a machine with local swap might contain:rootfs 192.1.2.3:/rootfs/myclient
hostname myclient.mydomainEnsure that your NFS server has exported the root (and swap if
applicable) filesystems to your client, and that the client has
root access to these filesystems A typical
/etc/exports file on FreeBSD might look
like:/rootfs/myclient -maproot=0:0 myclient.mydomain
/swapfs -maproot=0:0 myclient.mydomainAnd on HP-UX:/rootfs/myclient -root=myclient.mydomain
/swapfs -root=myclient.mydomainNFSswapping overIf you are swapping over NFS (completely diskless
configuration) create a swap file for your client using
dd. If your swapfs
command has the arguments /swapfs and
the size 20000 as in the example above, the swapfile for
myclient will be called
/swapfs/swap.X.X.X.X
where X.X.X.X is the client's IP
address, e.g.:&prompt.root; dd if=/dev/zero of=/swapfs/swap.192.1.2.4 bs=1k count=20000Also, the client's swap space might contain sensitive
information once swapping starts, so make sure to restrict read
and write access to this file to prevent unauthorized
access:&prompt.root; chmod 0600 /swapfs/swap.192.1.2.4Unpack the root filesystem in the directory the client will
use for its root filesystem (/rootfs/myclient
in the example above).On HP-UX systems: The server should be running HP-UX 9.04
or later for HP9000/800 series machines. Prior versions do not
allow the creation of device files over NFS.When extracting /dev in
/rootfs/myclient, beware that some
systems (HPUX) will not create device files that FreeBSD is
happy with. You may have to go to single user mode on the
first bootup (press control-c during the bootup phase), cd
/dev and do a sh ./MAKEDEV
all from the client to fix this.Run netboot.com on the client or make an
EPROM from the netboot.rom fileUsing Shared / and /usr
FilesystemsAlthough this is not an officially sanctioned or supported way
of doing this, some people report that it works quite well. If
anyone has any suggestions on how to do this cleanly, please tell
&a.doc;.Compiling Netboot for Specific SetupsNetboot can be compiled to support NE1000/2000 cards by changing
the configuration in
/sys/i386/boot/netboot/Makefile. See the
comments at the top of this file.ISDNA good resource for information on ISDN technology and hardware is
Dan Kegel's ISDN
Page.A quick simple road map to ISDN follows:If you live in Europe you might want to investigate the ISDN card
section.If you are planning to use ISDN primarily to connect to the
Internet with an Internet Provider on a dial-up non-dedicated basis,
you might look into Terminal Adapters. This will give you the
most flexibility, with the fewest problems, if you change
providers.If you are connecting two LANs together, or connecting to the
Internet with a dedicated ISDN connection, you might consider
the stand alone router/bridge option.Cost is a significant factor in determining what solution you will
choose. The following options are listed from least expensive to most
expensive.HellmuthMichaelisContributed by ISDN CardsISDNcardsFreeBSD's ISDN implementation supports only the DSS1/Q.931
(or Euro-ISDN) standard using passive cards. Starting with
FreeBSD 4.4, some active cards are supported where the firmware
also supports other signaling protocols; this also includes the
first supported Primary Rate (PRI) ISDN card.Isdn4bsd allows you to connect
to other ISDN routers using either IP over raw HDLC or by using
synchronous PPP: either by using kernel PPP with isppp, a
modified sppp driver, or by using userland &man.ppp.8;. By using
userland &man.ppp.8;, channel bonding of two or more ISDN
B-channels is possible. A telephone answering machine
application is also available as well as many utilities such as
a software 300 Baud modem.Some growing number of PC ISDN cards are supported under
FreeBSD and the reports show that it is successfully used all
over Europe and in many other parts of the world.The passive ISDN cards supported are mostly the ones with
the Infineon (formerly Siemens) ISAC/HSCX/IPAC ISDN chipsets,
but also ISDN cards with chips from Cologne Chip (ISA bus only),
PCI cards with Winbond W6692 chips, some cards with the
Tiger300/320/ISAC chipset combinations and some vendor specific
chipset based cards such as the AVM Fritz!Card PCI V.1.0 and the
AVM Fritz!Card PnP.Currently the active supported ISDN cards are the AVM B1
(ISA and PCI) BRI cards and the AVM T1 PCI PRI cards.For documentation on isdn4bsd,
have a look at /usr/share/examples/isdn/
directory on your FreeBSD system or at the homepage of
isdn4bsd which also has pointers to hints, erratas and
much more documentation such as the isdn4bsd
handbook.In case you are interested in adding support for a
different ISDN protocol, a currently unsupported ISDN PC card or
otherwise enhancing isdn4bsd, please
get in touch with hm@freebsd.org.For questions regarding the installation, configuration
and troubleshooting isdn4bsd, a
majordomo maintained mailing list is available. To join, send
mail to majordomo@FreeBSD.org and specify:subscribe freebsd-isdnin the body of your message.ISDN Terminal AdaptersTerminal adapters(TA), are to ISDN what modems are to regular
phone lines.modemMost TA's use the standard hayes modem AT command set, and can be
used as a drop in replacement for a modem.A TA will operate basically the same as a modem except connection
and throughput speeds will be much faster than your old modem. You
will need to configure PPP exactly the same
as for a modem setup. Make sure you set your serial speed as high as
possible.PPPThe main advantage of using a TA to connect to an Internet
Provider is that you can do Dynamic PPP. As IP address space becomes
more and more scarce, most providers are not willing to provide you
with a static IP anymore. Most stand-alone routers are not able to
accommodate dynamic IP allocation.TA's completely rely on the PPP daemon that you are running for
their features and stability of connection. This allows you to
upgrade easily from using a modem to ISDN on a FreeBSD machine, if you
already have PPP setup. However, at the same time any problems you
experienced with the PPP program and are going to persist.If you want maximum stability, use the kernel PPP option, not the user-land iijPPP.The following TA's are know to work with FreeBSD.Motorola BitSurfer and Bitsurfer ProAdtranMost other TA's will probably work as well, TA vendors try to make
sure their product can accept most of the standard modem AT command
set.The real problem with external TA's is like modems you need a good
serial card in your computer.You should read the FreeBSD Serial
Hardware tutorial for a detailed understanding of
serial devices, and the differences between asynchronous and
synchronous serial ports.A TA running off a standard PC serial port (asynchronous) limits
you to 115.2Kbs, even though you have a 128Kbs connection. To fully
utilize the 128Kbs that ISDN is capable of, you must move the TA to a
synchronous serial card.Do not be fooled into buying an internal TA and thinking you have
avoided the synchronous/asynchronous issue. Internal TA's simply have
a standard PC serial port chip built into them. All this will do, is
save you having to buy another serial cable, and find another empty
electrical socket.A synchronous card with a TA is at least as fast as a stand-alone
router, and with a simple 386 FreeBSD box driving it, probably more
flexible.The choice of sync/TA v.s. stand-alone router is largely a
religious issue. There has been some discussion of this in
the mailing lists. I suggest you search the archives for
the complete discussion.Stand-alone ISDN Bridges/RoutersISDNstand-alone bridges/routersISDN bridges or routers are not at all specific to FreeBSD
or any other operating system. For a more complete
description of routing and bridging technology, please refer
to a Networking reference book.In the context of this page, the terms router and bridge will
be used interchangeably.As the cost of low end ISDN routers/bridges comes down, it
will likely become a more and more popular choice. An ISDN
router is a small box that plugs directly into your local
Ethernet network, and manages its own connection to the other
bridge/router. It has built in software to communicate via
PPP and other popular protocols.A router will allow you much faster throughput that a
standard TA, since it will be using a full synchronous ISDN
connection.The main problem with ISDN routers and bridges is that
interoperability between manufacturers can still be a problem.
If you are planning to connect to an Internet provider, you
should discuss your needs with them.If you are planning to connect two LAN segments together,
such as your home LAN to the office LAN, this is the simplest
lowest
maintenance solution. Since you are buying the equipment for
both sides of the connection you can be assured that the link
will work.For example to connect a home computer or branch office
network to a head office network the following setup could be
used.Branch Office or Home Network10 base 2Network uses a bus based topology with 10 base 2
Ethernet ("thinnet"). Connect router to network cable with
AUI/10BT transceiver, if necessary.---Sun workstation
|
---FreeBSD box
|
---Windows 95 (Do not admit to owning it)
|
Stand-alone router
|
ISDN BRI line10 Base 2 EthernetIf your home/branch office is only one computer you can use a
twisted pair crossover cable to connect to the stand-alone router
directly.Head Office or Other LAN10 base TNetwork uses a star topology with 10 base T Ethernet
("Twisted Pair"). -------Novell Server
| H |
| ---Sun
| |
| U ---FreeBSD
| |
| ---Windows 95
| B |
|___---Stand-alone router
|
ISDN BRI lineISDN Network DiagramOne large advantage of most routers/bridges is that they allow you
to have 2 separate independent PPP connections to
2 separate sites at the same time. This is not
supported on most TA's, except for specific (usually expensive) models
that
have two serial ports. Do not confuse this with channel bonding, MPP,
etc.This can be very useful feature if, for example, you have an
dedicated ISDN connection at your office and would like to
tap into it, but do not want to get another ISDN line at work. A router
at the office location can manage a dedicated B channel connection
(64Kbps) to the Internet and use the other B channel for a
separate data connection. The second B channel can be used for
dial-in, dial-out or dynamically bonding (MPP, etc.) with the first
B channel for more bandwidth.IPX/SPXAn Ethernet bridge will also allow you to transmit more than just
IP traffic. You can also send IPX/SPX or whatever other protocols you
use.BillSwingleWritten by EricOgrenEnhanced by UdoErdelhoffNIS/YPWhat Is It?NISSolarisHP-UXAIXLinuxNetBSDOpenBSDNIS, which stands for Network Information Services, was
developed by Sun Microsystems to centralize administration of Unix
(originally SunOS) systems. It has now essentially become an
industry standard; all major Unix systems (Solaris, HP-UX, AIX, Linux,
NetBSD, OpenBSD, FreeBSD, etc) support NIS.yellow pagesNISNIS was formerly known as Yellow Pages, but because of
trademark issues, Sun changed the name. The old term (and yp) is
still often seen and used.NISdomainsIt is a RPC-based client/server system that allows a group
of machines within an NIS domain to share a common set of
configuration files. This permits a system administrator to set
up NIS client systems with only minimal configuration data and
add, remove or modify configuration data from a single
location.Windows NTIt is similar to Windows NT's domain system; although the
internal implementation of the two are not at all similar,
the basic functionality can be compared.Terms/Processes You Should KnowThere are several terms and several important user processes
that you will come across when
attempting to implement NIS on FreeBSD, whether you are trying to
create an NIS server or act an NIS client:portmapTermDescriptionNIS domainnameAn NIS master server and all of its clients
(including its slave servers) have a NIS
domainname. Similar to an NT domain name, the NIS
domainname does not have anything to do with DNS.portmapMust be running in order to enable RPC (Remote
Procedure Call, a network protocol used by NIS). If
portmap is not running, it will be
impossible to run an NIS server, or to act as an NIS
client.ypbind
- “binds” an NIS client to its NIS
+ binds an NIS client to its NIS
server. It will take the NIS domainname from the
system, and using RPC, connect to the
server. ypbind is the core of
client-server communication in an NIS environment; if
ypbind dies on a client machine, it
will not be able to access the NIS server.ypservShould only be running on NIS servers, is the NIS
server process itself. If &man.ypserv.8; dies, then the
server will no longer be able to respond to NIS requests
(hopefully, there is a slave server to take over for
it). There are some implementations of NIS (but not the
FreeBSD one), that do not try to reconnect to another
server if the server it used before dies. Often, the
only thing that helps in this case is to restart the
server process (or even the whole server) or the
ypbind process on the client.
rpc.yppasswddAnother process that should only be running on
NIS master servers, is a daemon that will allow NIS
clients to change their NIS passwords. If this daemon
is not running, users will have to login to the NIS
master server and change their passwords there.How Does It Work?There are three types of hosts in an NIS environment: master
servers, slave servers, and clients. Servers act as a central
repository for host configuration information. Master servers
hold the authoritative copy of this information, while slave
servers mirror this information for redundancy. Clients rely on
the servers to provide this information to them.Information in many files can be shared in this manner. The
master.passwd, group,
and hosts files are commonly shared via NIS.
Whenever a process on a client needs information that would
normally be found in these files locally, it makes a query to the
NIS server that it is bound to instead.Machine TypesNISmaster serverA NIS master server.
This server, analogous to a Windows
NT primary domain controller, maintains the files used by all
of the NIS clients. The passwd,
group, and other various files used by the
NIS clients live on the master server.It is possible for one machine to be an NIS
master server for more than one NIS domain. However, this will
not be covered in this introduction, which assumes a relatively
small-scale NIS environment.NISslave serverNIS slave servers.
Similar to NT's backup domain
controllers, NIS slave servers maintain copies of the NIS
master's data files. NIS slave servers provide the redundancy,
which is needed in important environments. They also help
to balance the load of the master server: NIS Clients always
attach to the NIS server whose response they get first, and
this includes slave-server-replies.NISclientNIS clients. NIS clients, like most
NT workstations, authenticate against the NIS server (or the NT
domain controller in the NT Workstation case) to log on.Using NIS/YPThis section will deal with setting up a sample NIS
environment.This section assumes that you are running FreeBSD 3.3
or later. The instructions given here will
probably work for any version of FreeBSD greater
than 3.0, but there are no guarantees that this is
true.PlanningLet us assume that you are the administrator of a small
university lab. This lab, which consists of 15 FreeBSD machines,
currently has no centralized point of administration; each machine
has its own /etc/passwd and
/etc/master.passwd. These files are kept in
sync with each other only through manual intervention;
currently, when you add a user to the lab, you must run
adduser on all 15 machines.
Clearly, this has to change, so you have decided to convert the
lab to use NIS, using two of the machines as servers.Therefore, the configuration of the lab now looks something
like:Machine nameIP addressMachine roleellington10.0.0.2NIS mastercoltrane10.0.0.3NIS slavebasie10.0.0.4Faculty workstationbird10.0.0.5Client machinecli[1-11]10.0.0.[6-17]Other client machinesIf you are setting up a NIS scheme for the first time, it
is a good idea to think through how you want to go about it. No
matter what the size of your network, there are a few decisions
that need to be made.Choosing a NIS Domain NameNISdomainnameThis might not be the domainname that you
are used to. It is more accurately called the
NIS domainname. When a client broadcasts its
requests for info, it includes the name of the NIS domain
that it is part of. This is how multiple servers on one
network can tell which server should answer which request.
Think of the NIS domainname as the name for a group of hosts
that are related in some way.Some organizations choose to use their Internet domainname
for their NIS domainname. This is not recommended as it can
cause confusion when trying to debug network problems. The
NIS domainname should be unique within your network and it is
helpful if it describes the group of machines it represents.
For example, the Art department at Acme Inc. might be in the
"acme-art" NIS domain. For this example, assume you have
chosen the name test-domain.SunOSHowever, some operating systems (notably SunOS) use their
NIS domain name as their Internet domain name.
If one or more machines on your network have this restriction,
you must use the Internet domain name as
your NIS domain name.Physical Server RequirementsThere are several things to keep in mind when choosing a
machine to use as a NIS server. One of the unfortunate things
about NIS is the level of dependency the clients have on the
server. If a client cannot contact the server for its NIS
domain, very often the machine becomes unusable. The lack of
user and group information causes most systems to temporarily
freeze up. With this in mind you should make sure to choose a
machine that will not be prone to being rebooted regularly, or
one that might be used for development. The NIS server should
ideally be a stand alone machine whose sole purpose in life is
to be an NIS server. If you have a network that is not very
heavily used, it is acceptable to put the NIS server on a
machine running other services, just keep in mind that if the
NIS server becomes unavailable, it will affect
all of your NIS clients adversely.NIS Servers The canonical copies of all NIS information are stored on
a single machine called the NIS master server. The databases
used to store the information are called NIS maps. In FreeBSD,
these maps are stored in
/var/yp/[domainname] where
[domainname] is the name of the NIS domain
being served. A single NIS server can support several domains
at once, therefore it is possible to have several such
directories, one for each supported domain. Each domain will
have its own independent set of maps.NIS master and slave servers handle all NIS requests with
the ypserv daemon. ypserv
is responsible for receiving incoming requests from NIS clients,
translating the requested domain and map name to a path to the
corresponding database file and transmitting data from the
database back to the client.Setting Up a NIS Master ServerNISserver configurationSetting up a master NIS server can be relatively straight
forward, depending on your needs. FreeBSD comes with support
for NIS out-of-the-box. All you need is to add the following
lines to /etc/rc.conf, and FreeBSD will
do the rest for you.nisdomainname="test-domain"
This line will set the NIS domainname to
test-domain
upon network setup (e.g. after reboot).nis_server_enable="YES"
This will tell FreeBSD to start up the NIS server processes
when the networking is next brought up.nis_yppasswdd_enable="YES"
This will enable the rpc.yppasswdd
daemon, which, as mentioned above, will allow users to
change their NIS password from a client machine.Depending on your NIS setup, you may need to add
further entries. See the section about NIS servers
that are also NIS clients, below, for
details.Now, all you have to do is to run the command
/etc/netstart as superuser. It will
setup everything for you, using the values you defined in
/etc/rc.conf.Initializing the NIS MapsNISmapsThe NIS maps are database files,
that are kept in the /var/yp directory.
They are generated from configuration files in the
/etc directory of the NIS master, with one
exception: the /etc/master.passwd file.
This is for a good reason; you do not want to propagate
passwords to your root and other administrative accounts to
all the servers in the NIS domain. Therefore, before we
initialize the NIS maps, you should:&prompt.root; cp /etc/master.passwd /var/yp/master.passwd
&prompt.root; cd /var/yp
&prompt.root; vi master.passwdYou should remove all entries regarding system accounts
(bin, tty, kmem,
games, etc), as well as any accounts that you
do not want to be propagated to the NIS clients (for example
root and any other UID 0 (superuser) accounts).Make sure the
/var/yp/master.passwd is neither group
nor world readable (mode 600)! Use the
chmod command, if appropriate.Tru64 UnixWhen you have finished, it is time to initialize the NIS
maps! FreeBSD includes a script named
ypinit to do this for you
(see its manual page for more information). Note that this
script is available on most Unix Operating Systems, but not on all.
On Digital Unix/Compaq Tru64 Unix it is called
ypsetup.
Because we are generating maps for an NIS master, we are
going to pass the option to
ypinit.
To generate the NIS maps, assuming you already performed
the steps above, run:ellington&prompt.root; ypinit -m test-domain
Server Type: MASTER Domain: test-domain
Creating an YP server will require that you answer a few questions.
Questions will all be asked at the beginning of the procedure.
Do you want this procedure to quit on non-fatal errors? [y/n: n] n
Ok, please remember to go back and redo manually whatever fails.
If you don't, something might not work.
At this point, we have to construct a list of this domains YP servers.
rod.darktech.org is already known as master server.
Please continue to add any slave servers, one per line. When you are
done with the list, type a <control D>.
master server : ellington
next host to add: coltrane
next host to add: ^D
The current list of NIS servers looks like this:
ellington
coltrane
Is this correct? [y/n: y] y
[..output from map generation..]
NIS Map update completed.
ellington has been setup as an YP master server without any errors.ypinit should have created
/var/yp/Makefile from
/var/yp/Makefile.dist.
When created, this file assumes that you are operating
in a single server NIS environment with only FreeBSD
machines. Since test-domain has
a slave server as well, you must edit
/var/yp/Makefile:ellington&prompt.root; vi /var/yp/MakefileYou should comment out the line that says `NOPUSH =
"True"' (if it is not commented out already).Setting up a NIS Slave ServerNISconfiguring a slave serverSetting up an NIS slave server is even more simple than
setting up the master. Log on to the slave server and edit the
file /etc/rc.conf as you did before.
The only difference is that we now must use the
option when running ypinit.
The option requires the name of the NIS
master be passed to it as well, so our command line looks
like:coltrane&prompt.root; ypinit -s ellington test-domain
Server Type: SLAVE Domain: test-domain Master: ellington
Creating an YP server will require that you answer a few questions.
Questions will all be asked at the beginning of the procedure.
Do you want this procedure to quit on non-fatal errors? [y/n: n] n
Ok, please remember to go back and redo manually whatever fails.
If you don't, something might not work.
There will be no further questions. The remainder of the procedure
should take a few minutes, to copy the databases from ellington.
Transferring netgroup...
ypxfr: Exiting: Map successfully transferred
Transferring netgroup.byuser...
ypxfr: Exiting: Map successfully transferred
Transferring netgroup.byhost...
ypxfr: Exiting: Map successfully transferred
Transferring master.passwd.byuid...
ypxfr: Exiting: Map successfully transferred
Transferring passwd.byuid...
ypxfr: Exiting: Map successfully transferred
Transferring passwd.byname...
ypxfr: Exiting: Map successfully transferred
Transferring group.bygid...
ypxfr: Exiting: Map successfully transferred
Transferring group.byname...
ypxfr: Exiting: Map successfully transferred
Transferring services.byname...
ypxfr: Exiting: Map successfully transferred
Transferring rpc.bynumber...
ypxfr: Exiting: Map successfully transferred
Transferring rpc.byname...
ypxfr: Exiting: Map successfully transferred
Transferring protocols.byname...
ypxfr: Exiting: Map successfully transferred
Transferring master.passwd.byname...
ypxfr: Exiting: Map successfully transferred
Transferring networks.byname...
ypxfr: Exiting: Map successfully transferred
Transferring networks.byaddr...
ypxfr: Exiting: Map successfully transferred
Transferring netid.byname...
ypxfr: Exiting: Map successfully transferred
Transferring hosts.byaddr...
ypxfr: Exiting: Map successfully transferred
Transferring protocols.bynumber...
ypxfr: Exiting: Map successfully transferred
Transferring ypservers...
ypxfr: Exiting: Map successfully transferred
Transferring hosts.byname...
ypxfr: Exiting: Map successfully transferred
coltrane has been setup as an YP slave server without any errors.
Don't forget to update map ypservers on ellington.You should now have a directory called
/var/yp/test-domain. Copies of the NIS
master server's maps should be in this directory. You will
need to make sure that these stay updated. The following
/etc/crontab entries on your slave
servers should do the job:20 * * * * root /usr/libexec/ypxfr passwd.byname
21 * * * * root /usr/libexec/ypxfr passwd.byuidThese two lines force the slave to sync its maps with
the maps on the master server. Although these entries are
not mandatory, since the master server attempts to ensure
any changes to its NIS maps are communicated to its slaves
and because password information is vital to systems
depending on the server, it is a good idea to force the
updates. This is more important on busy networks where map
updates might not always complete.Now, run the command /etc/netstart on the
slave server as well, which again starts the NIS server.NIS Clients An NIS client establishes what is called a binding to a
particular NIS server using the
ypbind daemon.
ypbind checks the system's default
domain (as set by the domainname command),
and begins broadcasting RPC requests on the local network.
These requests specify the name of the domain for which
ypbind is attempting to establish a binding.
If a server that has been configured to serve the requested
domain receives one of the broadcasts, it will respond to
ypbind, which will record the server's
address. If there are several servers available (a master and
several slaves, for example), ypbind will
use the address of the first one to respond. From that point
on, the client system will direct all of its NIS requests to
that server. ypbind will
occasionally ping the server to make sure it is
still up and running. If it fails to receive a reply to one of
its pings within a reasonable amount of time,
ypbind will mark the domain as unbound and
begin broadcasting again in the hopes of locating another
server.Setting Up an NIS ClientNISclient configurationSetting up a FreeBSD machine to be a NIS client is fairly
straightforward.Edit the file /etc/rc.conf and
add the following lines in order to set the NIS domainname
and start ypbind upon network
startup:nisdomainname="test-domain"
nis_client_enable="YES"To import all possible password entries from the NIS
server, remove all user accounts from your
/etc/master.passwd file and use
vipw to add the following line to
the end of the file:+:::::::::This line will afford anyone with a valid account in
the NIS server's password maps an account. There are
many ways to configure your NIS client by changing this
line. See the netgroups
section below for more information.
For more detailed reading see O'Reilly's book on
Managing NFS and NIS.You should keep at least one local account (i.e.
not imported via NIS) in your
/etc/master.passwd and this
account should also be a member of the group
wheel. If there is something
wrong with NIS, this account can be used to log in
remotely, become root, and fix things.To import all possible group entries from the NIS
server, add this line to your
/etc/group file:+:*::After completing these steps, you should be able to run
ypcat passwd and see the NIS server's
passwd map.NIS SecurityIn general, any remote user can issue an RPC to &man.ypserv.8; and
retrieve the contents of your NIS maps, provided the remote user
knows your domainname. To prevent such unauthorized transactions,
&man.ypserv.8; supports a feature called securenets which can be used to
restrict access to a given set of hosts. At startup, &man.ypserv.8; will
attempt to load the securenets information from a file called
/var/yp/securenets.This path varies depending on the path specified with the
option. This file contains entries that
consist of a network specification and a network mask separated
by white space. Lines starting with # are
considered to be comments. A sample securenets file might look
like this:# allow connections from local host -- mandatory
127.0.0.1 255.255.255.255
# allow connections from any host
# on the 192.168.128.0 network
192.168.128.0 255.255.255.0
# allow connections from any host
# between 10.0.0.0 to 10.0.15.255
# this includes the machines in the testlab
10.0.0.0 255.255.240.0If &man.ypserv.8; receives a request from an address that matches one
of these rules, it will process the request normally. If the
address fails to match a rule, the request will be ignored and a
warning message will be logged. If the
/var/yp/securenets file does not exist,
ypserv will allow connections from any host.The ypserv program also has support for Wietse
Venema's
tcpwrapper package. This allows the
administrator to use the tcpwrapper configuration
files for access control instead of
/var/yp/securenets.While both of these access control mechanisms provide some
security, they, like the privileged port test, are
vulnerable to IP spoofing attacks. All
NIS-related traffic should be blocked at your firewall.Servers using /var/yp/securenets
may fail to serve legitimate NIS clients with archaic TCP/IP
implementations. Some of these implementations set all
host bits to zero when doing broadcasts and/or fail to
observe the subnet mask when calculating the broadcast
address. While some of these problems can be fixed by
changing the client configuration, other problems may force
the retirement of the client systems in question or the
abandonment of /var/yp/securenets.Using /var/yp/securenets on a
server with such an archaic implementation of TCP/IP is a
really bad idea and will lead to loss of NIS functionality
for large parts of your network.tcpwrapperThe use of the tcpwrapper
package increases the latency of your NIS server. The
additional delay may be long enough to cause timeouts in
client programs, especially in busy networks or with slow
NIS servers. If one or more of your client systems
suffers from these symptoms, you should convert the client
systems in question into NIS slave servers and force them
to bind to themselves.Barring Some Users from Logging OnIn our lab, there is a machine basie that is
supposed to be a faculty only workstation. We do not want to take this
machine out of the NIS domain, yet the passwd
file on the master NIS server contains accounts for both faculty and
students. What can we do?There is a way to bar specific users from logging on to a
machine, even if they are present in the NIS database. To do this,
all you must do is add
-username to the end of
the /etc/master.passwd file on the client
machine, where username is the username of
the user you wish to bar from logging in. This should preferably be
done using vipw, since vipw
will sanity check your changes to
/etc/master.passwd, as well as
automatically rebuild the password database when you
finish editing. For example, if we wanted to bar user
bill from logging on to basie
we would:basie&prompt.root; vipw[add -bill to the end, exit]
vipw: rebuilding the database...
vipw: done
basie&prompt.root; cat /etc/master.passwd
root:[password]:0:0::0:0:The super-user:/root:/bin/csh
toor:[password]:0:0::0:0:The other super-user:/root:/bin/sh
daemon:*:1:1::0:0:Owner of many system processes:/root:/sbin/nologin
operator:*:2:5::0:0:System &:/:/sbin/nologin
bin:*:3:7::0:0:Binaries Commands and Source,,,:/:/sbin/nologin
tty:*:4:65533::0:0:Tty Sandbox:/:/sbin/nologin
kmem:*:5:65533::0:0:KMem Sandbox:/:/sbin/nologin
games:*:7:13::0:0:Games pseudo-user:/usr/games:/sbin/nologin
news:*:8:8::0:0:News Subsystem:/:/sbin/nologin
man:*:9:9::0:0:Mister Man Pages:/usr/share/man:/sbin/nologin
bind:*:53:53::0:0:Bind Sandbox:/:/sbin/nologin
uucp:*:66:66::0:0:UUCP pseudo-user:/var/spool/uucppublic:/usr/libexec/uucp/uucico
xten:*:67:67::0:0:X-10 daemon:/usr/local/xten:/sbin/nologin
pop:*:68:6::0:0:Post Office Owner:/nonexistent:/sbin/nologin
nobody:*:65534:65534::0:0:Unprivileged user:/nonexistent:/sbin/nologin
+:::::::::
-bill
basie&prompt.root;UdoErdelhoffContributed by Using NetgroupsnetgroupsThe method shown in the previous section works reasonably
well if you need special rules for a very small number of
users and/or machines. On larger networks, you
will forget to bar some users from logging
onto sensitive machines, or you may even have to modify each
machine separately, thus losing the main benefit of NIS,
centralized administration.The NIS developers' solution for this problem is called
netgroups. Their purpose and semantics
can be compared to the normal groups used by Unix file
systems. The main differences are the lack of a numeric id
and the ability to define a netgroup by including both user
accounts and other netgroups.Netgroups were developed to handle large, complex networks
with hundreds of users and machines. On one hand, this is
a Good Thing if you are forced to deal with such a situation.
On the other hand, this complexity makes it almost impossible to
explain netgroups with really simple examples. The example
used in the remainder of this section demonstrates this
problem.Let us assume that your successful introduction of NIS in
your laboratory caught your superiors' interest. Your next
job is to extend your NIS domain to cover some of the other
machines on campus. The two tables contain the names of the
new users and new machines as well as brief descriptions of
them.User Name(s)Descriptionalpha, betaNormal employees of the IT departmentcharlie, deltaThe new apprentices of the IT departmentecho, foxtrott, golf, ...Ordinary employeesable, baker, ...The current internsMachine Name(s)Descriptionwar, death, famine, pollutionYour most important servers. Only the IT
employees are allowed to log onto these
machines.pride, greed, envy, wrath, lust, slothLess important servers. All members of the IT
department are allowed to login onto these machines.one, two, three, four, ...Ordinary workstations. Only the
real employees are allowed to use
these machines.trashcanA very old machine without any critical data.
Even the intern is allowed to use this box.If you tried to implement these restrictions by separately
blocking each user, you would have to add one
-user line to each system's
passwd
for each user who is not allowed to login onto that system.
If you forget just one entry, you could be in trouble. It may
be feasible to do this correctly during the initial setup,
however you will eventually forget to add
the lines for new users during day-to-day operations. After
all, Murphy was an optimist.Handling this situation with netgroups offers several
advantages. Each user need not be handled separately;
you assign a user to one or more netgroups and allow or forbid
logins for all members of the netgroup. If you add a new
machine, you will only have to define login restrictions for
netgroups. If a new user is added, you will only have to add
the user to one or more netgroups. Those changes are
independent of each other; no more for each combination
of user and machine do... If your NIS setup is planned
carefully, you will only have to modify exactly one central
configuration file to grant or deny access to machines.The first step is the initialization of the NIS map
netgroup. FreeBSD's &man.ypinit.8; does not create this map by
default, but its NIS implementation will support it once it has
been created. To create an empty map, simply typeellington&prompt.root; vi /var/yp/netgroupand start adding content. For our example, we need at
least four netgroups: IT employees, IT apprentices, normal
employees and interns.IT_EMP (,alpha,test-domain) (,beta,test-domain)
IT_APP (,charlie,test-domain) (,delta,test-domain)
USERS (,echo,test-domain) (,foxtrott,test-domain) \
(,golf,test-domain)
INTERNS (,able,test-domain) (,baker,test-domain)IT_EMP, IT_APP etc.
are the names of the netgroups. Each bracketed group adds
one or more user accounts to it. The three fields inside a
group are:The name of the host(s) where the following items are
valid. If you do not specify a hostname, the entry is
valid on all hosts. If you do specify a hostname, you
will enter a realm of darkness, horror and utter confusion.The name of the account that belongs to this
netgroup.The NIS domain for the account. You can import
accounts from other NIS domains into your netgroup if you
are one of unlucky fellows with more than one NIS
domain.Each of these fields can contain wildcards. See
&man.netgroup.5; for details.netgroupsNetgroup names longer than 8 characters should not be
used, especially if you have machines running other
operating systems within your NIS domain. The names are
case sensitive; using capital letters for your netgroup
names is an easy way to distinguish between user, machine
and netgroup names.Some NIS clients (other than FreeBSD) cannot handle
netgroups with a large number of entries. For example, some
older versions of SunOS start to cause trouble if a netgroup
contains more than 15 entries. You can
circumvent this limit by creating several sub-netgroups with
15 users or less and a real netgroup that consists of the
sub-netgroups:BIGGRP1 (,joe1,domain) (,joe2,domain) (,joe3,domain) [...]
BIGGRP2 (,joe16,domain) (,joe17,domain) [...]
BIGGRP3 (,joe31,domain) (,joe32,domain)
BIGGROUP BIGGRP1 BIGGRP2 BIGGRP3You can repeat this process if you need more than 225
users within a single netgroup.Activating and distributing your new NIS map is
easy:ellington&prompt.root; cd /var/yp
ellington&prompt.root; makeThis will generate the three NIS maps
netgroup,
netgroup.byhost and
netgroup.byuser. Use &man.ypcat.1; to
check if your new NIS maps are available:ellington&prompt.user; ypcat -k netgroup
ellington&prompt.user; ypcat -k netgroup.byhost
ellington&prompt.user; ypcat -k netgroup.byuserThe output of the first command should resemble the
contents of /var/yp/netgroup. The second
command will not produce output if you have not specified
host-specific netgroups. The third command can be used to
get the list of netgroups for a user.The client setup is quite simple. To configure the server
war, you only have to start
&man.vipw.8; and replace the line+:::::::::with+@IT_EMP:::::::::Now, only the data for the users defined in the netgroup
IT_EMP is imported into
war's password database and only
these users are allowed to login.Unfortunately, this limitation also applies to the ~
function of the shell and all routines converting between user
names and numerical user ids. In other words,
cd ~user will not work,
ls -l will show the numerical id instead of
the username and find . -user joe -print will
fail with No such user. To fix this, you will
have to import all user entries without allowing them
to login onto your servers.This can be achieved by adding another line to
/etc/master.passwd. This line should
contain +:::::::::/sbin/nologin, meaning
Import all entries but replace the shell with
/sbin/nologin in the imported
entries. You can replace any field
in the passwd entry by placing a default value in your
/etc/master.passwd.Make sure that the line
+:::::::::/sbin/nologin is placed after
+@IT_EMP:::::::::. Otherwise, all user
accounts imported from NIS will have /sbin/nologin as their
login shell.After this change, you will only have to change one NIS
map if a new employee joins the IT department. You could use
a similar approach for the less important servers by replacing
the old +::::::::: in their local version
of /etc/master.passwd with something like
this:+@IT_EMP:::::::::
+@IT_APP:::::::::
+:::::::::/sbin/nologinThe corresponding lines for the normal workstations
could be:+@IT_EMP:::::::::
+@USERS:::::::::
+:::::::::/sbin/nologinAnd everything would be fine until there is a policy
change a few weeks later: The IT department starts hiring
interns. The IT interns are allowed to use the normal
workstations and the less important servers; and the IT
apprentices are allowed to login onto the main servers. You
add a new netgroup IT_INTERN, add the new IT interns to this
netgroup and start to change the config on each and every
machine... As the old saying goes: Errors in
centralized planning lead to global mess.NIS' ability to create netgroups from other netgroups can
be used to prevent situations like these. One possibility
is the creation of role-based netgroups. For example, you
could create a netgroup called
BIGSRV to define the login
restrictions for the important servers, another netgroup
called SMALLSRV for the less
important servers and a third netgroup called
USERBOX for the normal
workstations. Each of these netgroups contains the netgroups
that are allowed to login onto these machines. The new
entries for your NIS map netgroup should look like this:BIGSRV IT_EMP IT_APP
SMALLSRV IT_EMP IT_APP ITINTERN
USERBOX IT_EMP ITINTERN USERSThis method of defining login restrictions works
reasonably well if you can define groups of machines with
identical restrictions. Unfortunately, this is the exception
and not the rule. Most of the time, you will need the ability
to define login restrictions on a per-machine basis.Machine-specific netgroup definitions are the other
possibility to deal with the policy change outlined above. In
this scenario, the /etc/master.passwd of
each box contains two lines starting with ``+''. The first of
them adds a netgroup with the accounts allowed to login onto
this machine, the second one adds all other accounts with
/sbin/nologin as shell. It is a good
idea to use the ALL-CAPS version of the machine name as the
name of the netgroup. In other words, the lines should look
like this:+@BOXNAME:::::::::
+:::::::::/sbin/nologinOnce you have completed this task for all your machines,
you will not have to modify the local versions of
/etc/master.passwd ever again. All
further changes can be handled by modifying the NIS map. Here
is an example of a possible netgroup map for this
scenario with some additional goodies.# Define groups of users first
IT_EMP (,alpha,test-domain) (,beta,test-domain)
IT_APP (,charlie,test-domain) (,delta,test-domain)
DEPT1 (,echo,test-domain) (,foxtrott,test-domain)
DEPT2 (,golf,test-domain) (,hotel,test-domain)
DEPT3 (,india,test-domain) (,juliet,test-domain)
ITINTERN (,kilo,test-domain) (,lima,test-domain)
D_INTERNS (,able,test-domain) (,baker,test-domain)
#
# Now, define some groups based on roles
USERS DEPT1 DEPT2 DEPT3
BIGSRV IT_EMP IT_APP
SMALLSRV IT_EMP IT_APP ITINTERN
USERBOX IT_EMP ITINTERN USERS
#
# And a groups for a special tasks
# Allow echo and golf to access our anti-virus-machine
SECURITY IT_EMP (,echo,test-domain) (,golf,test-domain)
#
# machine-based netgroups
# Our main servers
WAR BIGSRV
FAMINE BIGSRV
# User india needs access to this server
POLLUTION BIGSRV (,india,test-domain)
#
# This one is really important and needs more access restrictions
DEATH IT_EMP
#
# The anti-virus-machine mentioned above
ONE SECURITY
#
# Restrict a machine to a single user
TWO (,hotel,test-domain)
# [...more groups to follow]If you are using some kind of database to manage your user
accounts, you should be able to create the first part of the
map with your database's report tools. This way, new users
will automatically have access to the boxes.One last word of caution: It may not always be advisable
to use machine-based netgroups. If you are deploying a couple
dozen or even hundreds of identical machines for student labs,
you should use role-based netgroups instead of machine-based
netgroups to keep the size of the NIS map within reasonable
limits.Important Things to RememberThere are still a couple of things that you will need to do
differently now that you are in an NIS environment.Every time you wish to add a user to the lab, you
must add it to the master NIS server only,
and you must remember to rebuild the NIS
maps. If you forget to do this, the new user will
not be able to login anywhere except on the NIS master.
For example, if we needed to add a new user
- “jsmith” to the lab, we would:
+ jsmith to the lab, we would:
&prompt.root; pw useradd jsmith
&prompt.root; cd /var/yp
&prompt.root; make test-domainYou could also run adduser jsmith instead
of pw useradd jsmith.Keep the administration accounts out of the NIS
maps. You do not want to be propagating administrative
accounts and passwords to machines that will have users that
should not have access to those accounts.Keep the NIS master and slave
secure, and minimize their downtime.
If somebody either hacks or simply turns off
these machines, they have effectively rendered many people without
the ability to login to the lab.This is the chief weakness of any centralized administration
system, and it is probably the most important weakness. If you do
not protect your NIS servers, you will have a lot of angry
users!NIS v1 Compatibility FreeBSD's ypserv has some support
for serving NIS v1 clients. FreeBSD's NIS implementation only
uses the NIS v2 protocol, however other implementations include
support for the v1 protocol for backwards compatibility with older
systems. The ypbind daemons supplied
with these systems will try to establish a binding to an NIS v1
server even though they may never actually need it (and they may
persist in broadcasting in search of one even after they receive a
response from a v2 server). Note that while support for normal
client calls is provided, this version of ypserv does not handle
v1 map transfer requests; consequently, it cannot be used as a
master or slave in conjunction with older NIS servers that only
support the v1 protocol. Fortunately, there probably are not any
such servers still in use today.NIS Servers that are also NIS Clients Care must be taken when running ypserv in a multi-server
domain where the server machines are also NIS clients. It is
generally a good idea to force the servers to bind to themselves
rather than allowing them to broadcast bind requests and possibly
become bound to each other. Strange failure modes can result if
one server goes down and others are dependent upon on it.
Eventually all the clients will time out and attempt to bind to
other servers, but the delay involved can be considerable and the
failure mode is still present since the servers might bind to each
other all over again.You can force a host to bind to a particular server by running
ypbind with the
flag. If you do not want to do this manually each time you
reboot your NIS server, you can add the following lines to
your /etc/rc.conf:nis_client_enable="YES" # run client stuff as well
nis_client_flags="-S NIS domain,server"See &man.ypbind.8; for further information.libscrypt v.s. libdescryptNIScrypto libraryOne of the most common issues that people run into when trying
to implement NIS is crypt library compatibility. If your NIS
server is using the DES crypt libraries, it will only support
clients that are using DES as well. To check which one your server
and clients are using look at the symlinks in
/usr/lib. If the machine is configured to
use the DES libraries, it will look something like this:&prompt.user; ls -l /usr/lib/*crypt*
lrwxrwxrwx 1 root wheel 13 Jul 15 08:55 libcrypt.a@ -> libdescrypt.a
lrwxrwxrwx 1 root wheel 14 Jul 15 08:55 libcrypt.so@ -> libdescrypt.so
lrwxrwxrwx 1 root wheel 16 Jul 15 08:55 libcrypt.so.2@ -> libdescrypt.so.2
lrwxrwxrwx 1 root wheel 15 Jul 15 08:55 libcrypt_p.a@ -> libdescrypt_p.a
-r--r--r-- 1 root wheel 13018 Nov 8 14:27 libdescrypt.a
lrwxr-xr-x 1 root wheel 16 Nov 8 14:27 libdescrypt.so@ -> libdescrypt.so.2
-r--r--r-- 1 root wheel 12965 Nov 8 14:27 libdescrypt.so.2
-r--r--r-- 1 root wheel 14750 Nov 8 14:27 libdescrypt_p.aIf the machine is configured to use the standard FreeBSD MD5
crypt libraries they will look something like this:&prompt.user; ls -l /usr/lib/*crypt*
lrwxrwxrwx 1 root wheel 13 Jul 15 08:55 libcrypt.a@ -> libscrypt.a
lrwxrwxrwx 1 root wheel 14 Jul 15 08:55 libcrypt.so@ -> libscrypt.so
lrwxrwxrwx 1 root wheel 16 Jul 15 08:55 libcrypt.so.2@ -> libscrypt.so.2
lrwxrwxrwx 1 root wheel 15 Jul 15 08:55 libcrypt_p.a@ -> libscrypt_p.a
-r--r--r-- 1 root wheel 6194 Nov 8 14:27 libscrypt.a
lrwxr-xr-x 1 root wheel 14 Nov 8 14:27 libscrypt.so@ -> libscrypt.so.2
-r--r--r-- 1 root wheel 7579 Nov 8 14:27 libscrypt.so.2
-r--r--r-- 1 root wheel 6684 Nov 8 14:27 libscrypt_p.aIf you have trouble authenticating on an NIS client, this
is a pretty good place to start looking for possible problems.
If you want to deploy an NIS server for a heterogenous
network, you will probably have to use DES on all systems
because it is the lowest common standard.GregSutterWritten by DHCPWhat Is DHCP?Dynamic Host Configuration ProtocolDHCPInternet Software Consortium (ISC)DHCP, the Dynamic Host Configuration Protocol, describes
the means by which a system can connect to a network and obtain the
necessary information for communication upon that network. FreeBSD
uses the ISC (Internet Software Consortium) DHCP implementation, so
all implementation-specific information here is for use with the ISC
distribution.What this Section CoversThis section attempts to describe only the parts
of the DHCP system that are integrated with FreeBSD;
consequently, the server portions are not described. The DHCP
manual pages, in addition to the references below, are useful
resources.How It WorksUDPWhen dhclient, the DHCP client, is executed on
the client
machine, it begins broadcasting requests for configuration
information. By default, these requests are on UDP port 68. The
server replies on UDP 67, giving the client an IP address and
other relevant network information such as netmask, router, and
DNS servers. All of this information comes in the form of a DHCP
"lease" and is only valid for a certain time (configured by the
DHCP server maintainer). In this manner, stale IP addresses for
clients no longer connected to the network can be automatically
reclaimed.DHCP clients can obtain a great deal of information from
the server. An exhaustive list may be found in
&man.dhcp-options.5;.FreeBSD IntegrationFreeBSD fully integrates the ISC DHCP client,
dhclient. DHCP client support is provided
within both the installer and the base system, obviating the need
for detailed knowledge of network configurations on any network
that runs a DHCP server. dhclient has been
included in all FreeBSD distributions since 3.2.sysinstallDHCP is supported by sysinstall.
When configuring a network interface within sysinstall,
the first question asked is, "Do you want to try DHCP
configuration of this interface?" Answering affirmatively will
execute dhclient, and if successful, will fill
in the network configuration information automatically.There are two things you must do to have your system use
DHCP upon startup:DHCPrequirementsMake sure that the bpf
device is compiled into your kernel. To do this, add
pseudo-device bpf to your kernel
configuration file, and rebuild the kernel. For more
information about building kernels, see .The bpf device is already
part of the GENERIC kernel that is
supplied with FreeBSD, so if you do not have a custom
kernel, you should not need to create one in order to get
DHCP working.For those who are particularly security conscious,
you should be warned that bpf
is also the device that allows packet sniffers to work
correctly (although they still have to be run as
root). bpfis required to use DHCP, but if
you are very sensitive about security, you probably
should not add bpf to your
kernel in the expectation that at some point in the
future you will be using DHCP.Edit your /etc/rc.conf to
include the following:ifconfig_fxp0="DHCP"Be sure to replace fxp0 with the
designation for the interface that you wish to dynamically
configure.If you are using a different location for
dhclient, or if you wish to pass additional
flags to dhclient, also include the
following (editing as necessary):dhcp_program="/sbin/dhclient"
dhcp_flags=""DHCPserverThe DHCP server, dhcpd, is included
as part of the isc-dhcp2 port in the ports
collection. This port contains the full ISC DHCP distribution,
consisting of client, server, relay agent and documentation.
FilesDHCPconfiguration files/etc/dhclient.confdhclient requires a configuration file,
/etc/dhclient.conf. Typically the file
contains only comments, the defaults being reasonably sane. This
configuration file is described by the &man.dhclient.conf.5;
manual page./sbin/dhclientdhclient is statically linked and
resides in /sbin. The &man.dhclient.8;
manual page gives more information about
dhclient./sbin/dhclient-scriptdhclient-script is the FreeBSD-specific
DHCP client configuration script. It is described in
&man.dhclient-script.8;, but should not need any user
modification to function properly./var/db/dhclient.leasesThe DHCP client keeps a database of valid leases in this
file, which is written as a log. &man.dhclient.leases.5;
gives a slightly longer description.Further ReadingThe DHCP protocol is fully described in
RFC 2131.
An informational resource has also been set up at
dhcp.org.ChernLeeContributed by DNSOverviewBINDFreeBSD utilizes, by default, a version of BIND (Berkeley
Internet Name Domain), which is the most common implementation of the
DNS protocol. DNS is the protocol through which names are mapped to
IP addresses, and vice versa. For example, a query for
www.FreeBSD.org
will receive a reply with the IP address of The FreeBSD Project's
web server, whereas, a query for ftp.FreeBSD.org
will return the IP
address of the corresponding FTP machine. Likewise, the opposite can
happen. A query for an IP address can resolve its hostname. It is
not necessary to run a name server to perform DNS lookups on a system.
DNSDNS is coordinated across the Internet through a somewhat
complex system of authoritative root name servers, and other
smaller-scale name servers who host and cache individual domain
information.
This document refers to BIND 8.x, as it is the stable version
used in FreeBSD. BIND 9.x in FreeBSD can be installed through
the net/bind9 port.
RFC1034 and RFC1035 dictates the DNS protocol.
Currently, BIND is maintained by the
Internet Software Consortium (www.isc.org)TerminologyTo understand this document, some terms related to DNS must be
understood.TermDefinitionforward DNSmapping of hostnames to IP addressesoriginrefers to the domain covered for the particular zone
filenamed, bind, name servercommon names for the BIND name server package within
FreeBSDresolverresolvera system process through which a
machine queries a name server for zone informationreverse DNSreverse DNSthe opposite of forward DNS, mapping of IP addresses to
hostnamesroot zoneroot zoneliterally, a ., refers to the
root, or beginning zone. All zones fall under this, as
do all files in fall under the root directory. It is
the beginning of the Internet zone hierarchy.zoneEach individual domain, subdomain, or area dictated by
DNSzonesexamplesExamples of zones:
. is the root zoneorg. is a zone under the root zonefoobardomain.org is a zone under the org. zonefoo.foobardomain.org. is a subdomain, a zone under the
foobardomain.org. zone
1.2.3.in-addr.arpa is a zone referencing all IP addresses
which fall under the 3.2.1.* IP space.
As one can see, the more specific part of a hostname appears to
its left. For example, foobardomain.org. is more
specific than org., as org. is
more specific than the root zone. The layout of each part of
a hostname is much like a filesystem: the /dev
directory falls within the root, and so on.Reasons to Run a Name ServerName servers usually come in two forms: an authoritative
name server, and a caching name server.An authoritative name server is needed when:one wants to serve DNS information to the
world, replying authoritatively to queries.a domain, such as foobardomain.org, is
registered and IP addresses need to be assigned to hostnames
under it.an IP address block requires reverse DNS entries (IP to
hostname).a backup name server, called a slave, must reply to queries
when the primary is down or inaccessible.A caching name server is needed when:a local DNS server may cache and respond more quickly
then querying an outside name server.a reduction in overall network traffic is desired. (DNS
traffic has been measured to account for 5% or more of total
Internet traffic)When one queries for www.FreeBSD.org, the
resolver usually queries the uplink ISP's name server, and retrieves
the reply. With a local, caching DNS server, the query only has to
be made once to the outside world by the caching DNS server. Every
additional query will not have to look to the outside of the local
network, since the information is cached locally.How It WorksIn FreeBSD, the BIND daemon is called
named for obvious reasons.FileDescriptionnamedthe BIND daemonndcname daemon control program/etc/namedbdirectory where BIND zone information resides/etc/namedb/named.confdaemon configuration file
Zone files are usually contained within the
/etc/namedb
directory, and contain the DNS zone information
served by the name server.
Starting BINDBINDstarting
Since BIND is installed by default, configuring it all is
relatively simple.
To ensure the named daemon is started at boot, put the following
modifications in /etc/rc.conf:
named_enable="YES"To start the daemon manually (after configuring it)&prompt.root; ndc startConfiguration FilesBINDconfiguration filesmake-localhostBe sure to:
&prompt.root; cd /etc/namedb
&prompt.root; sh make-localhostto properly create the local reverse DNS zone file in
/etc/namedb/localhost.rev.
/etc/namedb/named.conf// $FreeBSD$
//
// Refer to the named(8) manual page for details. If you are ever going
// to setup a primary server, make sure you've understood the hairy
// details of how DNS is working. Even with simple mistakes, you can
// break connectivity for affected parties, or cause huge amount of
// useless Internet traffic.
options {
directory "/etc/namedb";
// In addition to the "forwarders" clause, you can force your name
// server to never initiate queries of its own, but always ask its
// forwarders only, by enabling the following line:
//
// forward only;
// If you've got a DNS server around at your upstream provider, enter
// its IP address here, and enable the line below. This will make you
// benefit from its cache, thus reduce overall DNS traffic in the
Internet.
/*
forwarders {
127.0.0.1;
};
*/
Just as the comment says, to benefit from an uplink's cache,
forwarders can be enabled here. Under normal
circumstances, a name server will recursively query the Internet
looking at certain name servers until it finds the answer it is
looking for. Having this enabled will have it query the uplink's
name server (or name server provided) first, taking advantage of
its cache. If the uplink name server in question is a heavily
trafficked, fast name server, enabling this may be worthwhile.
127.0.0.1 will not work here.
Change this IP address to a name server at your uplink. /*
* If there is a firewall between you and name servers you want
* to talk to, you might need to uncomment the query-source
* directive below. Previous versions of BIND always asked
* questions using port 53, but BIND 8.1 uses an unprivileged
* port by default.
*/
// query-source address * port 53;
/*
* If running in a sandbox, you may have to specify a different
* location for the dumpfile.
*/
// dump-file "s/named_dump.db";
};
// Note: the following will be supported in a future release.
/*
host { any; } {
topology {
127.0.0.0/8;
};
};
*/
// Setting up secondaries is way easier and the rough picture for this
// is explained below.
//
// If you enable a local name server, don't forget to enter 127.0.0.1
// into your /etc/resolv.conf so this server will be queried first.
// Also, make sure to enable it in /etc/rc.conf.
zone "." {
type hint;
file "named.root";
};
zone "0.0.127.IN-ADDR.ARPA" {
type master;
file "localhost.rev";
};
zone
"0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.IP6.INT" {
type master;
file "localhost.rev";
};
// NB: Do not use the IP addresses below, they are faked, and only
// serve demonstration/documentation purposes!
//
// Example secondary config entries. It can be convenient to become
// a secondary at least for the zone where your own domain is in. Ask
// your network administrator for the IP address of the responsible
// primary.
//
// Never forget to include the reverse lookup (IN-ADDR.ARPA) zone!
// (This is the first bytes of the respective IP address, in reverse
// order, with ".IN-ADDR.ARPA" appended.)
//
// Before starting to setup a primary zone, better make sure you fully
// understand how DNS and BIND works, however. There are sometimes
// unobvious pitfalls. Setting up a secondary is comparably simpler.
//
// NB: Don't blindly enable the examples below. :-) Use actual names
// and addresses instead.
//
// NOTE!!! FreeBSD runs bind in a sandbox (see named_flags in rc.conf).
// The directory containing the secondary zones must be write accessible
// to bind. The following sequence is suggested:
//
// mkdir /etc/namedb/s
// chown bind:bind /etc/namedb/s
// chmod 750 /etc/namedb/sFor more information on running BIND in a sandbox, see
Running named in a sandbox.
/*
zone "domain.com" {
type slave;
file "s/domain.com.bak";
masters {
192.168.1.1;
};
};
zone "0.168.192.in-addr.arpa" {
type slave;
file "s/0.168.192.in-addr.arpa.bak";
masters {
192.168.1.1;
};
};
*/In named.conf, these are examples of slave
entries for a forward and reverse zone.For each new zone served, a new zone entry must be added to
named.confFor example, the simplest zone entry for foobardomain.org can
look like:zone "foobardomain.org" {
type master;
file "foobardomain.org";
};The zone is a master, as indicated by the
statement, holding its zone information in
/etc/namedb/foobardomain.org indicated by
the statement.zone "foobardomain.org" {
type slave;
file "foobardomain.org";
};In the slave case, the zone information is transferred from
the master name server for the particular zone, and saved in the
file specified. If and when the master server dies or is
unreachable, the slave name server will have the transferred
zone information and will be able to serve it.Zone Files
An example master zone file for foobardomain.org
(existing within /etc/namedb/foobardomain.org)
is as follows:
$TTL 3600
foobardomain.org. IN SOA ns1.foobardomain.org. admin.foobardomain.org. (
5 ; Serial
10800 ; Refresh
3600 ; Retry
604800 ; Expire
86400 ) ; Minimum TTL
; DNS Servers
@ IN NS ns1.foobardomain.org.
@ IN NS ns2.foobardomain.org.
; Machine Names
localhost IN A 127.0.0.1
ns1 IN A 3.2.1.2
ns2 IN A 3.2.1.3
mail IN A 3.2.1.10
@ IN A 3.2.1.30
; Aliases
www IN CNAME @
; MX Record
@ IN MX 10 mail.foobardomain.org.
Note that every hostname ending in a . is an
exact hostname, whereas everything without a trailing
. is referenced to the origin. For example,
www is translated into www +
origin. In our fictitious zone file, our origin
is foobardomain.org., so
www would translate to
www.foobardomain.org.
The format of a zone file follows:
recordname IN recordtype valueDNSrecords
The most commonly used DNS records:
SOAstart of zone authorityNSan authoritative name serverAA host addressCNAMEthe canonical name for an aliasMXmail exchangePTRa domain name pointer (used in reverse DNS)
foobardomain.org. IN SOA ns1.foobardomain.org. admin.foobardomain.org. (
5 ; Serial
10800 ; Refresh after 3 hours
3600 ; Retry after 1 hour
604800 ; Expire after 1 week
86400 ) ; Minimum TTL of 1 dayfoobardomain.org.the domain name, also the origin for this
zone file.ns1.foobardomain.org.the primary/authoritative name server for this
zoneadmin.foobardomain.org.the responsible person for this zone,
email address with @
replaced. (admin@foobardomain.org becomes
admin.foobardomain.org)5the serial number of the file. this
must be incremented each time the zone file is modified.
Nowadays, many admins prefer a
yyyymmddrr format for the serial
number. 2001041002 would mean last modified 04/10/2001,
the latter 02 being the second time the zone file has
been modified this day. The serial number is important
as it alerts slave name servers for a zone when it is
updated.
@ IN NS ns1.foobardomain.org.
This is an NS entry. Every name server that is going to reply
authoritatively for the zone must have one of these entries.
The @ as seen here could have been
foobardomain.org.
The @ translates to the origin.
localhost IN A 127.0.0.1
ns1 IN A 3.2.1.2
ns2 IN A 3.2.1.3
mail IN A 3.2.1.10
@ IN A 3.2.1.30
The A record indicates machine names. As seen above,
ns1.foobardomain.org would resolve to
3.2.1.2. Again, the origin symbol, @, is
used here, thus meaning foobardomain.org
would resolve to 3.2.1.30.
www IN CNAME @
The canonical name record is usually used for giving aliases
to a machine. In the example, www is
aliased to the machine addressed to the origin, or
foobardomain.org (3.2.1.30).
CNAMEs can be used to provide alias
hostnames, or round robin one hostname among multiple
machines.
@ IN MX 10 mail.foobardomain.org.
The MX record indicates which mail servers are responsible
for handling incoming mail for the zone.
mail.foobardomain.org is the hostname of the mail server,
and 10 being the priority of that mail server.
One can have several mail servers, with priorities of 3, 2,
1. A mail server attempting to deliver to foobardomain.org
would first try the highest priority MX, then the second
highest, etc, until the mail can be properly delivered.
For in-addr.arpa zone files (reverse DNS), the same format is
used, except with PTR entries instead of
A or CNAME.
$TTL 3600
1.2.3.in-addr.arpa. IN SOA ns1.foobardomain.org. admin.foobardomain.org. (
5 ; Serial
10800 ; Refresh
3600 ; Retry
604800 ; Expire
3600 ) ; Minimum
@ IN NS ns1.foobardomain.org.
@ IN NS ns2.foobardomain.org.
2 IN PTR ns1.foobardomain.org.
3 IN PTR ns2.foobardomain.org.
10 IN PTR mail.foobardomain.org.
30 IN PTR foobardomain.org.
This file gives the proper IP address to hostname mappings of our above
fictitious domain.
Caching Name ServerBINDcaching name server
A caching name server is a name server that is not
authoritative for any zones. It simply asks queries of its own,
and remembers them for later use. To set one up, just configure
the name server as usual, omitting any inclusions of zones.
MikeMakonnenContributed by Running named in a SandboxBINDrunning in a sandboxchrootFor added security you may want to run &man.named.8; in a
sandbox. This will reduce the potential damage should it be
compromised. If you include a sandbox directory in its command
line, named will &man.chroot.8;
into that directory immediately upon finishing processing its
command line. It is also a good idea to have named run as a
non-privileged user in the sandbox. The default FreeBSD install
contains a user bind with group bind. If we wanted the sandbox in
the /etc/namedb/sandbox directory the command
line for named would look like this:
&prompt.root; /usr/sbin/named -u bind -g bind -t /etc/namedb/sandbox <path_to_named.conf> The following steps should be taken in order to
successfully run named in a sandbox. Throughout the following
discussion we will assume the path to your sandbox is
/etc/namedb/sandboxCreate the sandbox directory:
/etc/namedb/sandboxCreate other necessary directories off of the sandbox
directory: etc and
var/runcopy /etc/localtime to
sandbox/etcmake bind:bind the owner of all files and directories in
the sandbox:
&prompt.root; chown -R bind:bind /etc/namedb/sandbox&prompt.root; chmod -R 750 /etc/namedb/sandboxThere are some issues you need to be aware of when running
named in a sandbox.Your &man.named.conf.5; file and all your zone files must
be in the sandbox
sandbox/etc/localtime is needed
in order to have the correct time for your time zone in
log messages. &man.named.8; will write its process id to a file in
sandbox/var/runThe Unix socket used for communication by the &man.ndc.8;
utility will be created in
sandbox/var/runWhen using the &man.ndc.8; utility you need to specify the
location of the Unix socket created in the sandbox, by
&man.named.8;, by using the -c switch:
&prompt.root; ndc -c /etc/namedb/sandbox/var/run/ndcIf you enable logging to file, the log files must be
in the sandbox&man.named.8; can be started in a sandbox properly, if the
following is in /etc/rc.conf:
named_flags="-u bind -g bind -t /etc/namedb/sandbox <path_to_named.conf>"How to Use the Name ServerIf setup properly, the name server should be accessible through
the network and locally. /etc/resolv.conf must
contain a name server entry with the local IP address so it will query the
local name server first.
To access it over the network, the machine must have the
name server's IP address set properly in its own name server
configuration options.
SecurityAlthough BIND is the most common implementation of DNS,
there is always the issue of security. Possible and
exploitable security holes are sometimes found.
It is a good idea to subscribe to CERT and
freebsd-announce
to stay up to date with the current Internet and FreeBSD security
issues.
If a problem arises, keeping sources up to date and having a
fresh build of named would not hurt.Further Reading
BIND/named manual pages: &man.ndc.8; &man.named.8; &man.named.conf.5;
Official ISC Bind
Page
BIND FAQO'Reilly
DNS and BIND 4th EditionRFC1034
- Domain Names - Concepts and FacilitiesRFC1035
- Domain Names - Implementation and SpecificationTomHukinsContributed by NTPOverviewNTP (Network Time Protocol)Over time, a computer's clock is prone to drift. As time
passes, the computer's clock becomes less accurate. NTP
(Network Time Protocol) is one way to ensure your clock is
right.Many Internet services rely on, or greatly benefit from,
computers' clocks being accurate. For example, a Web server
may receive requests to send a file if it has modified since a
certain time. Services such as &man.cron.8; run commands at a
given time. If the clock is inaccurate, these commands may
not run when expected.ntpdFreeBSD ships with the &man.ntpd.8; NTP server which can
be used to query other NTP servers to set the clock on your
machine or provide time services to others.Choosing appropriate NTP serversIn order to synchronize your clock, you will need to find
one or more NTP servers to use. Your network administrator or
ISP may have setup an NTP server for this purpose—check
their documentation to see if this is the case. There is a
list of
publicly accessible NTP servers which you can use to
find an NTP server near to you. Make sure you are aware of
the policy for any servers you choose, and ask for permission
if required.Choosing several unconnected NTP servers is a good idea in
case one of the servers you are using becomes unreachable or
its clock is unreliable. &man.ntpd.8; uses the responses it
receives from other servers intelligently—it will favor
unreliable servers less than reliable ones.Configuring your machineBasic ConfigurationntpdateIf you only wish to synchronize your clock when the
machine boots up, you can use &man.ntpdate.8;. This may be
appropriate for some desktop machines which are frequently
rebooted and only require infrequent synchronization, but
most machines should run &man.ntpd.8;.Using &man.ntpdate.8; at boot time is also a good idea
for machines that run &man.ntpd.8;. &man.ntpd.8; changes the
clock gradually, whereas &man.ntpdate.8; sets the clock, no
matter how great the difference between a machine's current
clock setting and the correct time.To enable &man.ntpdate.8; at boot time, add
ntpdate_enable="YES" to
/etc/rc.conf. You will also need to
specify all servers you wish to synchronize with and any
flags to be passed to &man.ntpdate.8; in
ntpdate_flags.ntp.confGeneral ConfigurationNTP is configured by the
/etc/ntp.conf file in the format
described in &man.ntp.conf.5;. Here is a simple
example:server ntplocal.example.com prefer
server timeserver.foobardomain.org
server ntp2a.example.net
driftfile /var/db/ntp.driftThe server option specifies which
servers are to be used, with one server listed on each line.
If a server is specified with the prefer
argument, as with ntplocal.example.com, that server is
preferred over other servers. A response from a preferred
server will be discarded if it differs significantly from
other servers' responses, otherwise it will be used without
any consideration to other responses. The
prefer argument is normally used for NTP
servers that are known to be highly accurate, such as those
with special time monitoring hardware.The driftfile option specifies which
file is used to store the system clock's frequency offset.
&man.ntpd.8; uses this to automatically compensate for the
clock's natural drift, allowing it to maintain a reasonably
correct setting even if it is cut off from all external time
sources for a period of time.The driftfile option specifies which
file is used to store information about previous responses
from the NTP servers you are using. This file contains
internal information for NTP. It should not be modified by
any other process.Controlling access to your serverBy default, your NTP server will be accessible to all
hosts on the Internet. The restrict
option in &man.ntp.conf.5; allows you to control which
machines can access your server.If you want to deny all machines from accessing your NTP
server, add the line restrict default ignore
to /etc/ntp.conf. If you only want to
allow machines within your own network to synchronize their
clocks with your server, but ensure they are not allowed to
configure the server or used as peers to synchronize
against, add restrict 192.168.1.0 mask 255.255.255.0 notrust nomodify notrap
instead, where 192.168.1.0 in
an IP address on your network and 255.255.255.0 is your network's
netmask./etc/ntp.conf can contain multiple
restrict options. For more details, see
the Access Control Support subsection of
&man.ntp.conf.5;.Running the NTP serverTo ensure the NTP server is started at boot time, add the
line xntpd_enable="YES" to
/etc/rc.conf. If you wish to pass
additional flags to &man.ntpd.8; edit the
xntpd_flags parameter in
/etc/rc.conf.To start the server without rebooting your machine, run
ntpd being sure to specify any additional
parameters from xntpd_flags in
/etc/rc.conf. For example:&prompt.root; ntpd -p /var/run/ntpd.pidUsing &man.ntpd.8; with a temporary Internet
connectionntpd does not need a permanent
connection to the Internet to function properly. However, if
you have a temporary connection that is configured to dial out
on demand, it is a good idea to prevent NTP traffic from
triggering a dial out or keeping the connection alive. If you
are using user PPP, you can use filter
directives in /etc/ppp/ppp.conf. For
example: set filter dial 0 deny udp src eq 123 # Prevent NTP traffic from initiating dial out
set filter dial 1 permit 0 0
set filter alive 0 deny udp src eq 123 # Prevent incoming NTP traffic from keeping the connection open
set filter alive 1 deny udp dst eq 123 # Prevent outgoing NTP traffic from keeping the connection open
set filter alive 2 permit 0/0 0/0For more details see the PACKET
FILTERING section in &man.ppp.8; and the examples in
/usr/share/examples/ppp/.Some Internet access providers block low-numbered ports,
preventing NTP from from functioning since replies never
reach your machine.Further InformationDocumentation for the NTP server can be found in
/usr/share/doc/ntp/ in HTML
format.ChernLeeContributed by Network Address TranslationOverviewnatdFreeBSD's Network Address Translation daemon, commonly known as
&man.natd.8; is a daemon that accepts incoming raw IP packets,
changes the source to the local machine and re-injects these packets
back into the outgoing IP packet stream. natd does this by changing
the source IP address and port such that when data is received back, it is
able to determine the original location of the data and forward it
back to its original requester.Internet connection sharingIP masqueradingThe most common use of NAT is to perform what is commonly known as
Internet Connection Sharing.SetupDue to the diminishing IP space in IPv4, and the increased number
of users on high-speed consumer lines such as cable or DSL, people are
in more and more need of an Internet Connection Sharing solution. The
ability to connect several computers online through one connection and
IP address makes &man.natd.8; a reasonable choice.Most commonly, a user has a machine connected to a cable or DSL
line with one IP address and wishes to use this one connected computer to
provide Internet access to several more over a LAN.To do this, the FreeBSD machine on the Internet must act as a
gateway. This gateway machine must have two NICs--one for connecting
to the Internet router, the other connecting to a LAN. All the
machines on the LAN are connected through a hub or switch. _______ __________ ________
| | | | | |
| Hub |-----| Client B |-----| Router |----- Internet
|_______| |__________| |________|
|
____|_____
| |
| Client A |
|__________|Network LayoutWith this setup, the machine without Internet access can use
the machine with access as a gateway to access the outside
world.kernelconfigurationConfigurationThe following options must be in the kernel configuration
file:options IPFIREWALL
options IPDIVERTAdditionally, at choice, the following may also be suitable:options IPFIREWALL_DEFAULT_TO_ACCEPT
options IPFIREWALL_VERBOSEThe following must be in /etc/rc.conf:gateway_enable="YES"
firewall_enable="YES"
firewall_type="OPEN"
natd_enable="YES"
natd_interface="fxp0"
natd_flags=""gateway_enable="YES"Sets up the machine to act as a gateway. Running
sysctl -w net.inet.ip.forwarding=1
would have the same effect.firewall_enable="YES"Enables the firewall rules in
/etc/rc.firewall at boot.firewall_type="OPEN"This specifies a predefined firewall ruleset that
allows anything in. See
/etc/rc.firewall for additional
types.natd_interface="fxp0"Indicates which interface to forward packets through.
(the interface connected to the Internet)natd_flags=""Any additional configuration options passed to
&man.natd.8; on boot.Having the previous options defined in
/etc/rc.conf would run
natd -interface fxp0 at boot. This can also
be run manually.Each machine and interface behind the LAN should be assigned IP address
numbers in the private network space as defined by
RFC 1918
and have a default gateway of the natd machine's internal IP address.For example, client a and b behind the LAN have IP addresses of 192.168.0.2
and 192.168.0.3, while the natd machine's LAN interface has an IP address of
192.168.0.1. Client a and b's default gateway must be set to that of
the natd machine, 192.168.0.1. The natd machine's external, or
Internet interface does not require any special modification for natd
to work.Port RedirectionThe drawback with natd is that the LAN clients are not accessible
from the Internet. Clients on the LAN can make outgoing connections to
the world but cannot receive incoming ones. This presents a problem
if trying to run Internet services on one of the LAN client machines.
A simple way around this is to redirect selected Internet ports on the
natd machine to a LAN client.
For example, an IRC server runs on Client A, and a web server runs
on Client B. For this to work properly, connections received on ports
6667 (irc) and 80 (web) must be redirected to the respective machines.
The -redirect_port must be passed to
&man.natd.8; with the proper options. The syntax is as follows: -redirect_port proto targetIP:targetPORT[-targetPORT]
[aliasIP:]aliasPORT[-aliasPORT]
[remoteIP[:remotePORT[-remotePORT]]]In the above example, the argument should be:
-redirect_port tcp 192.168.0.2:6667 6667
-redirect_port tcp 192.168.0.3:80 80
This will redirect the proper tcp ports to the
LAN client machines.
The -redirect_port argument can be used to indicate port
ranges over individual ports. For example, tcp
192.168.0.2:2000-3000 2000-3000 would redirect
all connections received on ports 2000 to 3000 to ports 2000
to 3000 on Client A.These options can be used when directly running
&man.natd.8; or placed within the
natd_flags="" option in
/etc/rc.conf.For further configuration options, consult &man.natd.8;Address Redirectionaddress redirectionAddress redirection is useful if several IP addresses are available, yet
they must be on one machine. With this, &man.natd.8; can assign each
LAN client its own external IP address. &man.natd.8; then rewrites outgoing
packets from the LAN clients with the proper external IP address and redirects
all traffic incoming on that particular IP address back to the specific LAN
client. This is also known as static NAT. For example, the IP addresses
128.1.1.1, 128.1.1.2, and 128.1.1.3 belong to the natd gateway
machine. 128.1.1.1 can be used as the natd gateway machine's external
IP address, while 128.1.1.2 and 128.1.1.3 are forwarded back to LAN
clients A and B.The -redirect_address syntax is as follows:localIPThe internal IP address of the LAN client.publicIPThe external IP address corresponding to the LAN client.In the example, this argument would read:Like -redirect_port, these arguments are also placed within
natd_flags of /etc/rc.conf. With address
redirection, there is no need for port redirection since all data
received on a particular IP address is redirected.The external IP addresses on the natd machine must be active and aliased
to the external interface. Look at &man.rc.conf.5; to do so.ChernLeeContributed by inetd Super-ServerOverview&man.inetd.8; is referred to as the Internet
Super-Server because it manages connections for several
daemons. Programs that provide network service are commonly
known as daemons. inetd serves as a
managing server for other daemons. When a connection is
received by inetd, it determines
which daemon the connection is destined for, spawns the
particular daemon and delegates the socket to it. Running one
instance of inetd reduces the overall
system load as compared to running each daemon individually in
stand-alone mode.Primarily, inetd is used to
spawn other daemons, but several trivial protocols are handled
directly, such as chargen,
auth, and
daytime.This section will cover the basics in configuring
inetd through its command-line
options and its configuration file,
/etc/inetd.conf.Settingsinetd is initialized through
the /etc/rc.conf system. The
inetd_enable option is set to
NO by default, but is often times turned on by
sysinstall with the medium security
profile. Placing:
inetd_enable="YES" or
inetd_enable="NO" into
/etc/rc.conf can enable or disable
inetd starting at boot time.Additionally, different command-line options can be passed
to inetd via the
inetd_flags option.Command-Line Optionsinetd sypnosis:-dTurn on debugging.-lTurn on logging of successful connections.-wTurn on TCP Wrapping for external services. (on by
default)-WTurn on TCP Wrapping for internal services which are
built in to inetd. (on by
default)-c maximumSpecify the default maximum number of simultaneous
invocations of each service; the default is unlimited.
May be overridden on a per-service basis with the
parameter.-C rateSpecify the default maximum number of times a
service can be invoked from a single IP address in one
minute; the default is unlimited. May be overridden on a
per-service basis with the
parameter.-R rateSpecify the maximum number of times a service can be
invoked in one minute; the default is 256. A rate of 0
allows an unlimited number of invocations.-aSpecify one specific IP address to bind to.
Alternatively, a hostname can be specified, in which case
the IPv4 or IPv6 address which corresponds to that
hostname is used. Usually a hostname is specified when
inetd is run inside a
&man.jail.8;, in which case the hostname corresponds to
the &man.jail.8; environment.When hostname specification is used and both IPv4
and IPv6 bindings are desired, one entry with the
appropriate protocol type for each binding is required for
each service in /etc/inetd.conf. For
example, a TCP-based service would need two entries, one
using ``tcp4'' for the protocol and the other using
``tcp6''.-pSpecify an alternate file in which to store the
process ID.These options can be passed to
inetd using the
inetd_flags option in
/etc/rc.conf. By default,
inetd_flags is set to -wW,
which turns on TCP wrapping for
inetd's internal and external
services. For novice users, these parameters usually do not need
to be modified or even entered in
/etc/rc.confAn external service is a daemon outside of
inetd, which is invoked when a
connection is received for it. On the other hand, an internal
service is one that inetd has the
facility of offering within itself.inetd.confConfiguration of inetd is
controlled through the /etc/inetd.conf
file.When a modification is made to
/etc/inetd.conf,
inetd can be forced to re-read its
configuration file by sending a HangUP signal to the
inetd process as shown:Sending inetd a HangUP Signal&prompt.root kill -HUP `cat /var/run/inetd.pid`Each line of the configuration file specifies an
individual daemon. Comments in the file are preceded by a
#. The format of
/etc/inetd.conf is as follows:service-name
socket-type
protocol
{wait|nowait}[/max-child[/max-connections-per-ip-per-minute]]
user[:group][/login-class]
server-program
server-program-argumentsAn example entry for the ftpd daemon
using IPv4:ftp stream tcp nowait root /usr/libexec/ftpd ftpd -lservice-nameThis is the service name of the particular daemon.
It must correspond to a service listed in
/etc/services. This determines which
port inetd must listen to. If
a new service is being created, it must be placed in
/etc/services
first.socket-typeEither stream,
dgram, raw, or
seqpacket. stream
must be used for connection-based, TCP daemons, while
dgram is used for daemons utilizing the
UDP transport protocol.protocolOne of the following:ProtocolExplanationtcp, tcp4TCP IPv4udp, udp4UDP IPv4tcp6TCP IPv6udp6UDP IPv6tcp46Both TCP IPv4 and v6udp46Both UDP IPv4 and v6{wait|nowait}[/max-child[/max-connections-per-ip-per-minute]] indicates whether the
daemon invoked from inetd is
able to handle its own socket or not.
socket types must use the wait
option, while stream socket daemons, which are usually
multi-threaded, should use .
usually hands off multiple sockets
to a single daemon, while spawns a
child daemon for each new socket.The maximum number of child daemons
inetd may spawn can be set using
the option. If a limit of ten
instances of a particular daemon is needed, a
/10 would be placed after
.In addition to another
option limiting the maximum connections from a single
place to a particular daemon can be enabled.
does
just this. A value of ten here would limit any particular
IP address connecting to a particular service to ten
attempts per minute. This is useful to prevent
intentional or unintentional resource consumption and
Denial of Service (DoS) attacks to a machine.In this field, or
is mandatory.
and
are
optional.A stream-type multi-threaded daemon without any
or
limits
would simply be: nowaitThe same daemon with a maximum limit of ten daemons
would read: nowait/10Additionally, the same setup with a limit of twenty
connections per IP address per minute and a maximum
total limit of ten child daemons would read:
nowait/10/20These options are all utilized by the default
settings of the fingerd daemon,
as seen here:finger stream tcp nowait/3/10 nobody /usr/libexec/fingerd fingerd -suserThe user is the username that the particular daemon
should run as. Most commonly, daemons run as the
root user. For security purposes, it is
common to find some servers running as the
daemon user, or the least privileged
nobody user.server-programThe full path of the daemon to be executed when a
connection is received. If the daemon is a service
provided by inetd internally,
then should be
used.server-program-argumentsThis works in conjunction with
by specifying the
arguments, starting with argv[0], passed to the daemon on
invocation. If mydaemon -d is
the command line, mydaemon -d would be
the value of .
Again, if the daemon is an internal service, use
here.SecurityDepending on the security profile chosen at install, many
of inetd's daemons may be enabled by
default. If there is no apparent need for a particular daemon,
disable it! Place a # in front of the daemon in
question, and send a hangup signal
to inetd.
Some daemons, such as fingerd, may
not be desired at all because they provide an attacker with too
much information.Some daemons are not security-conscious and have long, or
non-existent timeouts for connection attempts. This allows an
attacker to slowly send connections to a particular daemon, thus
saturating available resources. It may be a good idea to place
and
limitations on certain daemons.By default, TCP wrapping is turned on. Consult the
&man.hosts.access.5; manual page for more information on placing
TCP restrictions on various inetd
invoked daemons.Miscellaneousdaytime,
time,
echo,
discard,
chargen, and
auth are all internally provided
services of inetd.The auth service provides identity
(ident, identd) network services, and is configurable to a certain
degree.Consult the &man.inetd.8; manual page for more in-depth
information.
diff --git a/en_US.ISO8859-1/books/handbook/mirrors/chapter.sgml b/en_US.ISO8859-1/books/handbook/mirrors/chapter.sgml
index 30c09759bb..7cf6dee003 100644
--- a/en_US.ISO8859-1/books/handbook/mirrors/chapter.sgml
+++ b/en_US.ISO8859-1/books/handbook/mirrors/chapter.sgml
@@ -1,3824 +1,3824 @@
Obtaining FreeBSDCDROM PublishersRetail Boxed ProductsFreeBSD is available as a boxed product (FreeBSD CDs,
additional software, and printed documentation) from several
retailers:CompUSA
WWW: http://www.compusa.com/Frys Electronics
WWW: http://www.frys.com/Micro Center
WWW: http://www.microcenter.com/CD SetsFreeBSD CD sets are available from many online
retailers:Daemon News2672 Bayshore Parkway, Suite 610Mountain View, CA94043USA
Phone: +1 650 694-4949
Email: sales@daemonnews.org
WWW: http://www.bsdmall.com/Wind River Systems500 Wind River WayAlameda, CA94501USA
Phone: +1 510 749-2872
WWW: http://www.freebsdmall.com/DistributorsIf you are a reseller and want to carry FreeBSD CDROM products,
please contact a distributor :Cylogistics2672 Bayshore Parkway, Suite 610Mountain View, CA94043USA
Phone: +1 650 694-4949
Fax: +1 650 694-4953
Email: sales@cylogistics.com
WWW: http://www.cylogistics.com/Ingram Micro
WWW: http://www.ingrammicro.com/Navarre
WWW: http://www.navarre.com/DVD PublishersFreeBSD is available on DVD from:FreeBSD Services Ltd11 Lapwing CloseBicesterOX26 6XRUnited Kingdom
WWW: http://www.freebsd-services.com/FTP SitesThe official sources for FreeBSD are available via anonymous FTP
from:
ftp://ftp.FreeBSD.org/pub/FreeBSD/.
The FreeBSD mirror
sites database is more accurate than the mirror listing in the
Handbook, as it gets its information from the DNS rather than relying on
static lists of hosts.Additionally, FreeBSD is available via anonymous FTP from the
following mirror sites. If you choose to obtain FreeBSD via anonymous
FTP, please try to use a site near you.Argentina,
Australia,
Brazil,
Canada,
China,
Czech Republic,
Denmark,
Estonia,
Finland,
France,
Germany,
Hong Kong,
Hungary,
Ireland,
Israel,
Japan,
Korea,
Lithuania,
Netherlands,
New Zealand,
Poland,
Portugal,
Romania,
Russia,
Saudi Arabia,
South Africa,
Spain,
Slovak Republic,
Slovenia,
Sweden,
Taiwan,
Thailand,
UK,
Ukraine,
USA.ArgentinaIn case of problems, please contact the hostmaster
hostmaster@ar.FreeBSD.org for this domain.ftp://ftp.ar.FreeBSD.org/pub/FreeBSD/AustraliaIn case of problems, please contact the hostmaster
hostmaster@au.FreeBSD.org for this domain.ftp://ftp.au.FreeBSD.org/pub/FreeBSD/ftp://ftp2.au.FreeBSD.org/pub/FreeBSD/ftp://ftp3.au.FreeBSD.org/pub/FreeBSD/ftp://ftp4.au.FreeBSD.org/pub/FreeBSD/BrazilIn case of problems, please contact the hostmaster
hostmaster@br.FreeBSD.org for this domain.ftp://ftp.br.FreeBSD.org/pub/FreeBSD/ftp://ftp2.br.FreeBSD.org/pub/FreeBSD/ftp://ftp3.br.FreeBSD.org/pub/FreeBSD/ftp://ftp4.br.FreeBSD.org/pub/FreeBSD/ftp://ftp5.br.FreeBSD.org/pub/FreeBSD/ftp://ftp6.br.FreeBSD.org/pub/FreeBSD/ftp://ftp7.br.FreeBSD.org/pub/FreeBSD/CanadaIn case of problems, please contact the hostmaster
hostmaster@ca.FreeBSD.org for this domain.ftp://ftp.ca.FreeBSD.org/pub/FreeBSD/ChinaIn case of problems, please contact the hostmaster
phj@cn.FreeBSD.org for this domain.ftp://ftp.cn.FreeBSD.org/pub/FreeBSD/Czech RepublicIn case of problems, please contact the hostmaster
hostmaster@cz.FreeBSD.org for this domain.ftp://ftp.cz.FreeBSD.org/pub/FreeBSD/ Contact: calda@dzungle.ms.mff.cuni.czDenmarkIn case of problems, please contact the hostmaster
hostmaster@dk.FreeBSD.org for this domain.ftp://ftp.dk.FreeBSD.org/pub/FreeBSD/ftp://ftp2.dk.FreeBSD.org/pub/FreeBSD/ftp://ftp3.dk.FreeBSD.org/pub/FreeBSD/EstoniaIn case of problems, please contact the hostmaster
hostmaster@ee.FreeBSD.org for this domain.ftp://ftp.ee.FreeBSD.org/pub/FreeBSD/FinlandIn case of problems, please contact the hostmaster
hostmaster@fi.FreeBSD.org for this domain.ftp://ftp.fi.FreeBSD.org/pub/FreeBSD/FranceIn case of problems, please contact the hostmaster
hostmaster@fr.FreeBSD.org for this domain.ftp://ftp.fr.FreeBSD.org/pub/FreeBSD/ftp://ftp2.fr.FreeBSD.org/pub/FreeBSD/ftp://ftp3.fr.FreeBSD.org/pub/FreeBSD/ftp://ftp4.fr.FreeBSD.org/pub/FreeBSD/ftp://ftp5.fr.FreeBSD.org/pub/FreeBSD/ftp://ftp6.fr.FreeBSD.org/pub/FreeBSD/GermanyIn case of problems, please contact the mirror admins
de-bsd-hubs@de.FreeBSD.org for this domain.ftp://ftp.de.FreeBSD.org/pub/FreeBSD/ftp://ftp2.de.FreeBSD.org/pub/FreeBSD/ftp://ftp3.de.FreeBSD.org/pub/FreeBSD/ftp://ftp4.de.FreeBSD.org/pub/FreeBSD/ftp://ftp5.de.FreeBSD.org/pub/FreeBSD/ftp://ftp6.de.FreeBSD.org/pub/FreeBSD/ftp://ftp7.de.FreeBSD.org/pub/FreeBSD/Hong Kongftp://ftp.hk.super.net/pub/FreeBSD/ Contact: ftp-admin@HK.Super.NET.HungaryIn case of problems, please contact the hostmaster
mohacsi@ik.bme.hu for this domain.ftp://ftp.hu.FreeBSD.org/pub/FreeBSD/IrelandIn case of problems, please contact the hostmaster
hostmaster@ie.FreeBSD.org for this domain.ftp://ftp.ie.FreeBSD.org/pub/FreeBSD/IsraelIn case of problems, please contact the hostmaster
hostmaster@il.FreeBSD.org for this domain.ftp://ftp.il.FreeBSD.org/pub/FreeBSD/ftp://ftp2.il.FreeBSD.org/pub/FreeBSD/JapanIn case of problems, please contact the hostmaster
hostmaster@jp.FreeBSD.org for this domain.ftp://ftp.jp.FreeBSD.org/pub/FreeBSD/ftp://ftp2.jp.FreeBSD.org/pub/FreeBSD/ftp://ftp3.jp.FreeBSD.org/pub/FreeBSD/ftp://ftp4.jp.FreeBSD.org/pub/FreeBSD/ftp://ftp5.jp.FreeBSD.org/pub/FreeBSD/ftp://ftp6.jp.FreeBSD.org/pub/FreeBSD/KoreaIn case of problems, please contact the hostmaster
hostmaster@kr.FreeBSD.org for this domain.ftp://ftp.kr.FreeBSD.org/pub/FreeBSD/ftp://ftp2.kr.FreeBSD.org/pub/FreeBSD/ftp://ftp3.kr.FreeBSD.org/pub/FreeBSD/ftp://ftp4.kr.FreeBSD.org/pub/FreeBSD/ftp://ftp5.kr.FreeBSD.org/pub/FreeBSD/ftp://ftp6.kr.FreeBSD.org/pub/FreeBSD/LithuaniaIn case of problems, please contact the hostmaster
hostmaster@lt.FreeBSD.org for this domain.ftp://ftp.lt.FreeBSD.org/pub/FreeBSD/NetherlandsIn case of problems, please contact the hostmaster
hostmaster@nl.FreeBSD.org for this domain.ftp://ftp.nl.FreeBSD.org/pub/FreeBSD/New ZealandIn case of problems, please contact the hostmaster
hostmaster@nz.FreeBSD.org for this domain.ftp://ftp.nz.FreeBSD.org/pub/FreeBSD/PolandIn case of problems, please contact the hostmaster
hostmaster@pl.FreeBSD.org for this domain.ftp://ftp.pl.FreeBSD.org/pub/FreeBSD/PortugalIn case of problems, please contact the hostmaster
hostmaster@pt.FreeBSD.org for this domain.ftp://ftp.pt.FreeBSD.org/pub/FreeBSD/ftp://ftp2.pt.FreeBSD.org/pub/FreeBSD/RomaniaIn case of problems, please contact the hostmaster
hostmaster@ro.FreeBSD.org for this domain.ftp://ftp.ro.FreeBSD.org/pub/FreeBSD/RussiaIn case of problems, please contact the hostmaster
hostmaster@ru.FreeBSD.org for this domain.ftp://ftp.ru.FreeBSD.org/pub/FreeBSD/ftp://ftp2.ru.FreeBSD.org/pub/FreeBSD/ftp://ftp3.ru.FreeBSD.org/pub/FreeBSD/ftp://ftp4.ru.FreeBSD.org/pub/FreeBSD/Saudi ArabiaIn case of problems, please contact
ftpadmin@isu.net.saftp://ftp.isu.net.sa/pub/mirrors/ftp.freebsd.org/South AfricaIn case of problems, please contact the hostmaster
hostmaster@za.FreeBSD.org for this domain.ftp://ftp.za.FreeBSD.org/pub/FreeBSD/ftp://ftp2.za.FreeBSD.org/pub/FreeBSD/ftp://ftp3.za.FreeBSD.org/FreeBSD/Slovak RepublicIn case of problems, please contact the hostmaster
hostmaster@sk.FreeBSD.org for this domain.ftp://ftp.sk.FreeBSD.org/pub/FreeBSD/SloveniaIn case of problems, please contact the hostmaster
hostmaster@si.FreeBSD.org for this domain.ftp://ftp.si.FreeBSD.org/pub/FreeBSD/SpainIn case of problems, please contact the hostmaster
hostmaster@es.FreeBSD.org for this domain.ftp://ftp.es.FreeBSD.org/pub/FreeBSD/SwedenIn case of problems, please contact the hostmaster
hostmaster@se.FreeBSD.org for this domain.ftp://ftp.se.FreeBSD.org/pub/FreeBSD/ftp://ftp2.se.FreeBSD.org/pub/FreeBSD/ftp://ftp3.se.FreeBSD.org/pub/FreeBSD/TaiwanIn case of problems, please contact the hostmaster
hostmaster@tw.FreeBSD.org for this domain.ftp://ftp.tw.FreeBSD.org/pub/FreeBSD/ftp://ftp2.tw.FreeBSD.org/pub/FreeBSD/ftp://ftp3.tw.FreeBSD.org/pub/FreeBSD/ftp://ftp4.tw.FreeBSD.org/pub/FreeBSD/Thailandftp://ftp.nectec.or.th/pub/FreeBSD/ Contact: ftpadmin@ftp.nectec.or.th.Ukraineftp://ftp.ua.FreeBSD.org/pub/FreeBSD/ Contact: freebsd-mnt@lucky.net.UKIn case of problems, please contact the hostmaster
hostmaster@uk.FreeBSD.org for this domain.ftp://ftp.uk.FreeBSD.org/pub/FreeBSD/ftp://ftp2.uk.FreeBSD.org/pub/FreeBSD/ftp://ftp3.uk.FreeBSD.org/pub/FreeBSD/ftp://ftp4.uk.FreeBSD.org/pub/FreeBSD/ftp://ftp5.uk.FreeBSD.org/pub/FreeBSD/USAIn case of problems, please contact the hostmaster
hostmaster@FreeBSD.org for this domain.ftp://ftp2.FreeBSD.org/pub/FreeBSD/ftp://ftp3.FreeBSD.org/pub/FreeBSD/ftp://ftp4.FreeBSD.org/pub/FreeBSD/ftp://ftp5.FreeBSD.org/pub/FreeBSD/ftp://ftp6.FreeBSD.org/pub/FreeBSD/ftp://ftp7.FreeBSD.org/pub/FreeBSD/ftp://ftp8.FreeBSD.org/pub/FreeBSD/ftp://ftp9.FreeBSD.org/pub/os/FreeBSD/ftp://ftp10.FreeBSD.org/pub/FreeBSD/ftp://ftp11.FreeBSD.org/pub/FreeBSD/ftp://ftp12.FreeBSD.org/pub/FreeBSD/ftp://ftp13.FreeBSD.org/pub/FreeBSD/Anonymous CVSIntroductionAnonymous CVS (or, as it is otherwise known,
anoncvs) is a feature provided by the CVS
utilities bundled with FreeBSD for synchronizing with a remote
CVS repository. Among other things, it allows users of FreeBSD
to perform, with no special privileges, read-only CVS operations
against one of the FreeBSD project's official anoncvs servers.
To use it, one simply sets the CVSROOT
environment variable to point at the appropriate anoncvs server,
provides the well-known password anoncvs with the
cvs login command, and then uses the
&man.cvs.1; command to access it like any local
repository.While it can also be said that the CVSup and anoncvs
services both perform essentially the same function, there are
various trade-offs which can influence the user's choice of
synchronization methods. In a nutshell,
CVSup is much more efficient in its
usage of network resources and is by far the most technically
sophisticated of the two, but at a price. To use
CVSup, a special client must first be
installed and configured before any bits can be grabbed, and
then only in the fairly large chunks which
CVSup calls
collections.Anoncvs, by contrast, can be used
to examine anything from an individual file to a specific
program (like ls or grep)
by referencing the CVS module name. Of course,
anoncvs is also only good for
read-only operations on the CVS repository, so if it is your
intention to support local development in one repository shared
with the FreeBSD project bits then
CVSup is really your only
option.Using Anonymous CVSConfiguring &man.cvs.1; to use an Anonymous CVS repository
is a simple matter of setting the CVSROOT
environment variable to point to one of the FreeBSD project's
anoncvs servers. At the time of this
writing, the following servers are available:USA:
:pserver:anoncvs@anoncvs.FreeBSD.org:/home/ncvs
(Use cvs login and enter the password
anoncvs when prompted.)Since CVS allows one to check out virtually
any version of the FreeBSD sources that ever existed (or, in
some cases, will exist, you need to be
familiar with the revision () flag to
&man.cvs.1; and what some of the permissible values for it in
the FreeBSD Project repository are.There are two kinds of tags, revision tags and branch tags.
A revision tag refers to a specific revision. Its meaning stays
the same from day to day. A branch tag, on the other hand,
refers to the latest revision on a given line of development, at
any given time. Because a branch tag does not refer to a
specific revision, it may mean something different tomorrow than
it means today. contains revision tags that users
might be interested
in. Again, none of these are valid for the ports collection
since the ports collection does not have multiple
revisions.When you specify a branch tag, you normally receive the
latest versions of the files on that line of development. If
you wish to receive some past version, you can do so by
specifying a date with the flag.
See the &man.cvs.1; manual page for more details.ExamplesWhile it really is recommended that you read the manual page
for &man.cvs.1; thoroughly before doing anything, here are some
quick examples which essentially show how to use Anonymous
CVS:Checking Out Something from -CURRENT (&man.ls.1;) and
Deleting It Again:&prompt.user; setenv CVSROOT :pserver:anoncvs@anoncvs.FreeBSD.org:/home/ncvs
&prompt.user; cvs loginAt the prompt, enter the passwordanoncvs.
&prompt.user; cvs co ls
&prompt.user; cvs release -d ls
&prompt.user; cvs logoutChecking Out the Version of &man.ls.1; in the 3.X-STABLE
Branch:&prompt.user; setenv CVSROOT :pserver:anoncvs@anoncvs.FreeBSD.org:/home/ncvs
&prompt.user; cvs loginAt the prompt, enter the passwordanoncvs.
&prompt.user; cvs co -rRELENG_3 ls
&prompt.user; cvs release -d ls
&prompt.user; cvs logoutCreating a List of Changes (as unified diffs) to &man.ls.1;&prompt.user; setenv CVSROOT :pserver:anoncvs@anoncvs.FreeBSD.org:/home/ncvs
&prompt.user; cvs loginAt the prompt, enter the passwordanoncvs.
&prompt.user; cvs rdiff -u -rRELENG_3_0_0_RELEASE -rRELENG_3_4_0_RELEASE ls
&prompt.user; cvs logoutFinding Out What Other Module Names Can Be Used:&prompt.user; setenv CVSROOT :pserver:anoncvs@anoncvs.FreeBSD.org:/home/ncvs
&prompt.user; cvs loginAt the prompt, enter the passwordanoncvs.
&prompt.user; cvs co modules
&prompt.user; more modules/modules
&prompt.user; cvs release -d modules
&prompt.user; cvs logoutOther ResourcesThe following additional resources may be helpful in learning
CVS:CVS Tutorial from Cal Poly.Cyclic Software,
commercial maintainers of CVS.CVSWeb is
the FreeBSD Project web interface for CVS.Using CTMCTM is a method for keeping a
remote directory tree in sync with a central one. It has been
developed for usage with FreeBSD's source trees, though other
people may find it useful for other purposes as time goes by.
Little, if any, documentation currently exists at this time on the
process of creating deltas, so talk to &a.phk; for more
information should you wish to use CTM
for other things.Why Should I Use CTM?CTM will give you a local copy of
the FreeBSD source trees. There are a number of
- “flavors” of the tree available. Whether you wish
+ flavors of the tree available. Whether you wish
to track the entire CVS tree or just one of the branches,
CTM can provide you the information.
If you are an active developer on FreeBSD, but have lousy or
non-existent TCP/IP connectivity, or simply wish to have the
changes automatically sent to you,
CTM was made for you. You will need
to obtain up to three deltas per day for the most active
branches. However, you should consider having them sent by
automatic email. The sizes of the updates are always kept as
small as possible. This is typically less than 5K, with an
occasional (one in ten) being 10-50K and every now and then a
large 100K+ or more coming around.You will also need to make yourself aware of the various
caveats related to working directly from the development sources
rather than a pre-packaged release. This is particularly true
- if you choose the “current” sources. It is
+ if you choose the current sources. It is
recommended that you read Staying
current with FreeBSD.What Do I Need to Use
CTM?You will need two things: The CTM
program, and the initial deltas to feed it (to get up to
- “current” levels).
+ current levels).
The CTM program has been part of
FreeBSD ever since version 2.0 was released, and lives in
/usr/src/usr.sbin/CTM if you have a copy
of the source available.If you are running a pre-2.0 version of FreeBSD, you can
fetch the current CTM sources
directly from:ftp://ftp.FreeBSD.org/pub/FreeBSD/FreeBSD-current/src/usr.sbin/ctm/
- The “deltas” you feed
+ The deltas you feed
CTM can be had two ways, FTP or
email. If you have general FTP access to the Internet then the
following FTP sites support access to
CTM:ftp://ftp.FreeBSD.org/pub/FreeBSD/CTM/or see section mirrors.FTP the relevant directory and fetch the
README file, starting from there.If you wish to get your deltas via email:Send email to &a.majordomo; to subscribe to one of the
CTM distribution lists.
- “ctm-cvs-cur” supports the entire CVS tree.
- “ctm-src-cur” supports the head of the development
- branch. “ctm-src-2_2” supports the 2.2 release
+ ctm-cvs-cur supports the entire CVS tree.
+ ctm-src-cur supports the head of the development
+ branch. ctm-src-2_2 supports the 2.2 release
branch, etc.. (If you do not know how to subscribe yourself
using majordomo, send a message first containing the word
help — it will send you back usage
instructions.)When you begin receiving your CTM
updates in the mail, you may use the
ctm_rmail program to unpack and apply them.
You can actually use the ctm_rmail program
directly from a entry in /etc/aliases if
you want to have the process run in a fully automated fashion.
Check the ctm_rmail manual page for more
details.No matter what method you use to get the
CTM deltas, you should subscribe to
the ctm-announce@FreeBSD.org mailing list. In
the future, this will be the only place where announcements
concerning the operations of the
CTM system will be posted. Send an
email to &a.majordomo; with a single line of
subscribe ctm-announce to get added to the
list.Using CTM for the First
TimeBefore you can start using CTM
deltas, you will need to get to a starting point for the deltas
produced subsequently to it.First you should determine what you already have. Everyone
- can start from an “empty” directory. You must use
- an initial “Empty” delta to start off your
+ can start from an empty directory. You must use
+ an initial Empty delta to start off your
CTM supported tree. At some point it
- is intended that one of these “started” deltas be
+ is intended that one of these started deltas be
distributed on the CD for your convenience, however, this does
not currently happen.Since the trees are many tens of megabytes, you should
prefer to start from something already at hand. If you have a
-RELEASE CD, you can copy or extract an initial source from it.
This will save a significant transfer of data.
- You can recognize these “starter” deltas by the
+ You can recognize these starter deltas by the
X appended to the number
(src-cur.3210XEmpty.gz for instance). The
designation following the X corresponds to
- the origin of your initial “seed”.
+ the origin of your initial seed.
Empty is an empty directory. As a rule a
base transition from Empty is produced
every 100 deltas. By the way, they are large! 25 to 30
Megabytes of gzip'd data is common for the
XEmpty deltas.Once you have picked a base delta to start from, you will also
need all deltas with higher numbers following it.Using CTM in Your Daily
LifeTo apply the deltas, simply say:&prompt.root; cd /where/ever/you/want/the/stuff
&prompt.root; ctm -v -v /where/you/store/your/deltas/src-xxx.*CTM understands deltas which have
been put through gzip, so you do not need to
gunzip them first, this saves disk space.Unless it feels very secure about the entire process,
CTM will not touch your tree. To
verify a delta you can also use the flag and
CTM will not actually touch your
tree; it will merely verify the integrity of the delta and see
if it would apply cleanly to your current tree.There are other options to CTM
as well, see the manual pages or look in the sources for more
information.That is really all there is to it. Every time you get a new
delta, just run it through CTM to
keep your sources up to date.Do not remove the deltas if they are hard to download again.
You just might want to keep them around in case something bad
happens. Even if you only have floppy disks, consider using
fdwrite to make a copy.Keeping Your Local ChangesAs a developer one would like to experiment with and change
files in the source tree. CTM
supports local modifications in a limited way: before checking
for the presence of a file foo, it first
looks for foo.ctm. If this file exists,
CTM will operate on it instead of
foo.This behavior gives us a simple way to maintain local
changes: simply copy the files you plan to modify to the
corresponding file names with a .ctm
suffix. Then you can freely hack the code, while CTM keeps the
.ctm file up-to-date.Other Interesting CTM OptionsFinding Out Exactly What Would Be Touched by an
UpdateYou can determine the list of changes that
CTM will make on your source
repository using the option to
CTM.This is useful if you would like to keep logs of the
changes, pre- or post- process the modified files in any
manner, or just are feeling a tad paranoid.Making Backups Before UpdatingSometimes you may want to backup all the files that would
be changed by a CTM update.Specifying the option
causes CTM to backup all files that
would be touched by a given CTM
delta to backup-file.Restricting the Files Touched by an UpdateSometimes you would be interested in restricting the scope
of a given CTM update, or may be
interested in extracting just a few files from a sequence of
deltas.You can control the list of files that
CTM would operate on by specifying
filtering regular expressions using the
and options.For example, to extract an up-to-date copy of
lib/libc/Makefile from your collection of
saved CTM deltas, run the commands:&prompt.root; cd /where/ever/you/want/to/extract/it/
&prompt.root; ctm -e '^lib/libc/Makefile' ~ctm/src-xxx.*For every file specified in a
CTM delta, the
and options are applied in the order given
on the command line. The file is processed by
CTM only if it is marked as
eligible after all the and
options are applied to it.Future Plans for CTMTons of them:Use some kind of authentication into the CTM system, so
as to allow detection of spoofed CTM updates.Clean up the options to CTM,
they became confusing and counter intuitive.Miscellaneous StuffThere is a sequence of deltas for the
ports collection too, but interest has not
been all that high yet. Tell me if you want an email list for
that too and we will consider setting it up.CTM MirrorsCTM/FreeBSD is available via anonymous
FTP from the following mirror sites. If you choose to obtain CTM via
anonymous FTP, please try to use a site near you.In case of problems, please contact &a.phk;.California, Bay Area, official sourceftp://ftp.FreeBSD.org/pub/FreeBSD/development/CTM/Germany, Trierftp://ftp.uni-trier.de/pub/unix/systems/BSD/FreeBSD/CTM/South Africa, backup server for old deltasftp://ftp.za.FreeBSD.org/pub/FreeBSD/CTM/Taiwan/R.O.C, Chiayiftp://ctm.tw.FreeBSD.org/pub/FreeBSD/CTM/ftp://ctm2.tw.FreeBSD.org/pub/FreeBSD/CTM/ftp://ctm3.tw.FreeBSD.org/pub/freebsd/CTM/If you did not find a mirror near to you or the mirror is
incomplete, try FTP
search at http://ftpsearch.ntnu.no/ftpsearch.
FTP search is a great free archie server in Trondheim, Norway.Using CVSupIntroductionCVSup is a software package for
distributing and updating source trees from a master CVS
repository on a remote server host. The FreeBSD sources are
maintained in a CVS repository on a central development machine
in California. With CVSup, FreeBSD
users can easily keep their own source trees up to date.CVSup uses the so-called
pull model of updating. Under the pull
model, each client asks the server for updates, if and when they
are wanted. The server waits passively for update requests from
its clients. Thus all updates are instigated by the client.
The server never sends unsolicited updates. Users must either
run the CVSup client manually to get
an update, or they must set up a cron job to
run it automatically on a regular basis.The term CVSup, capitalized just
so, refers to the entire software package. Its main components
are the client cvsup which runs on each
user's machine, and the server cvsupd which
runs at each of the FreeBSD mirror sites.As you read the FreeBSD documentation and mailing lists, you
may see references to sup.
Sup was the predecessor of
CVSup, and it served a similar
purpose.CVSup is in used in much the
same way as sup and, in fact, uses configuration files which are
backward-compatible with sup's.
Sup is no longer used in the FreeBSD
project, because CVSup is both faster
and more flexible.InstallationThe easiest way to install CVSup
is to use the precompiled net/cvsup package
from the FreeBSD packages collection.
If you prefer to build CVSup from
source, you can use the net/cvsup
port instead. But be forewarned: the
net/cvsup port depends on the Modula-3
system, which takes a substantial amount of time and
disk space to download and build.If you do not know anything about
CVSup at all and want a
single package which will install it, set up the configuration
file and start the transfer via a pointy-clicky type of
interface, then get the cvsupit
package. Just hand it to &man.pkg.add.1; and it will lead you
through the configuration process in a menu-oriented
fashion.CVSup ConfigurationCVSup's operation is controlled
by a configuration file called the supfile.
There are some sample supfiles in the
directory /usr/share/examples/cvsup/.The information in a supfile answers
the following questions for cvsup:Which files do you
want to receive?Which versions of them
do you want?Where do you want to
get them from?Where do you want to
put them on your own machine?Where do you want to
put your status files?In the following sections, we will construct a typical
supfile by answering each of these
questions in turn. First, we describe the overall structure of
a supfile.A supfile is a text file. Comments
begin with # and extend to the end of the
line. Lines that are blank and lines that contain only
comments are ignored.Each remaining line describes a set of files that the user
wishes to receive. The line begins with the name of a
collection, a logical grouping of files defined by
the server. The name of the collection tells the server which
files you want. After the collection name come zero or more
fields, separated by white space. These fields answer the
questions listed above. There are two types of fields: flag
fields and value fields. A flag field consists of a keyword
standing alone, e.g., delete or
compress. A value field also begins with a
keyword, but the keyword is followed without intervening white
space by = and a second word. For example,
release=cvs is a value field.A supfile typically specifies more than
one collection to receive. One way to structure a
supfile is to specify all of the relevant
fields explicitly for each collection. However, that tends to
make the supfile lines quite long, and it
is inconvenient because most fields are the same for all of the
collections in a supfile.
CVSup provides a defaulting mechanism
to avoid these problems. Lines beginning with the special
pseudo-collection name *default can be used
to set flags and values which will be used as defaults for the
subsequent collections in the supfile. A
default value can be overridden for an individual collection, by
specifying a different value with the collection itself.
Defaults can also be changed or augmented in mid-supfile by
additional *default lines.With this background, we will now proceed to construct a
supfile for receiving and updating the main
source tree of FreeBSD-CURRENT.Which files do you want
to receive?The files available via CVSup
are organized into named groups called
collections. The collections that are
available are described in the following section. In this
example, we
wish to receive the entire main source tree for the FreeBSD
system. There is a single large collection
src-all which will give us all of that.
As a first step toward constructing our
supfile, we
simply list the collections, one per line (in this case,
only one line):src-allWhich version(s) of them
do you want?With CVSup, you can receive
virtually any version of the sources that ever existed.
That is possible because the
cvsupd server works directly from
the CVS repository, which contains all of the versions. You
specify which one of them you want using the
tag= and value
fields.Be very careful to specify any tag=
fields correctly. Some tags are valid only for certain
collections of files. If you specify an incorrect or
misspelled tag, CVSup
will delete files which you probably
do not want deleted. In particular, use only
tag=. for the
ports-* collections.The tag= field names a symbolic tag
in the repository. There are two kinds of tags, revision
tags and branch tags. A revision tag refers to a specific
revision. Its meaning stays the same from day to day. A
branch tag, on the other hand, refers to the latest revision
on a given line of development, at any given time. Because
a branch tag does not refer to a specific revision, it may
mean something different tomorrow than it means
today. contains branch tags that
users might be interested in. When specifying a tag in
CVSup's configuration file, it
must be prececeded with tag=
(RELENG_4 will become
tag=RELENG_4).
Keep in mind that only the tag=. is
relevant for the ports collection.Be very careful to type the tag name exactly as shown.
CVSup cannot distinguish
between valid and invalid tags. If you misspell the tag,
CVSup will behave as though you
had specified a valid tag which happens to refer to no
files at all. It will delete your existing sources in
that case.When you specify a branch tag, you normally receive the
latest versions of the files on that line of development.
If you wish to receive some past version, you can do so by
specifying a date with the value
field. The &man.cvsup.1; manual page explains how to do
that.For our example, we wish to receive FreeBSD-CURRENT. We
add this line at the beginning of our
supfile:*default tag=.There is an important special case that comes into play
if you specify neither a tag= field nor a
date= field. In that case, you receive
the actual RCS files directly from the server's CVS
repository, rather than receiving a particular version.
Developers generally prefer this mode of operation. By
maintaining a copy of the repository itself on their
systems, they gain the ability to browse the revision
histories and examine past versions of files. This gain is
achieved at a large cost in terms of disk space,
however.Where do you want to get
them from?We use the host= field to tell
cvsup where to obtain its updates. Any
of the CVSup mirror
sites will do, though you should try to select one
that is close to you in cyberspace. In this example we will
use a fictional FreeBSD distribution site,
cvsup666.FreeBSD.org:*default host=cvsup666.FreeBSD.orgYou will need to change the host to one that actually
exists before running CVSup.
On any particular run of
cvsup, you can override the host setting
on the command line, with .Where do you want to put
them on your own machine?The prefix= field tells
cvsup where to put the files it receives.
In this example, we will put the source files directly into
our main source tree, /usr/src. The
src directory is already implicit in
the collections we have chosen to receive, so this is the
correct specification:*default prefix=/usrWhere should
cvsup maintain its status files?The CVSup client maintains
certain status files in what
is called the base directory. These files
help CVSup to work more
efficiently, by keeping track of which updates you have
already received. We will use the standard base directory,
/usr/local/etc/cvsup:*default base=/usr/local/etc/cvsupThis setting is used by default if it is not specified
in the supfile, so we actually do not
need the above line.If your base directory does not already exist, now would
be a good time to create it. The cvsup
client will refuse to run if the base directory does not
exist.Miscellaneous supfile
settings:There is one more line of boiler plate that normally
needs to be present in the
supfile:*default release=cvs delete use-rel-suffix compressrelease=cvs indicates that the server
should get its information out of the main FreeBSD CVS
repository. This is virtually always the case, but there
are other possibilities which are beyond the scope of this
discussion.delete gives
CVSup permission to delete files.
You should always specify this, so that
CVSup can keep your source tree
fully up-to-date. CVSup is
careful to delete only those files for which it is
responsible. Any extra files you happen to have will be
left strictly alone.use-rel-suffix is ... arcane. If you
really want to know about it, see the &man.cvsup.1; manual
page. Otherwise, just specify it and do not worry about
it.compress enables the use of
gzip-style compression on the communication channel. If
your network link is T1 speed or faster, you probably should
not use compression. Otherwise, it helps
substantially.Putting it all together:Here is the entire supfile for our
example:*default tag=.
*default host=cvsup666.FreeBSD.org
*default prefix=/usr
*default base=/usr/local/etc/cvsup
*default release=cvs delete use-rel-suffix compress
src-allThe refuse FileAs mentioned above, CVSup uses
a pull method. Basically, this means that
you connect to the CVSup server, and
it says, Here's what you can download from
me..., and your client responds OK, I'll take
this, this, this, and this. In the default
configuration, the CVSup client will
take every file associated with the collection and tag you
chose in the configuration file. However, this is not always
what you want, especially if you are synching the doc, ports, or
www trees — most people cannot read four or five
languages, and therefore they do not need to download the
language-specific files. If you are
CVSuping the ports collection, you
can get around this by specifying each collection individually
(e.g., ports-astrology,
ports-biology, etc instead of simply
saying ports-all). However, since the doc
and www trees do not have language-specific collections, you
must use one of CVSup's many nifty
features; the refuse file.The refuse file essentially tells
CVSup that it should not take every
single file from a collection; in other words, it tells the
client to refuse certain files from the
server. The refuse file can be found (or, if you do not yet
have one, should be placed) in
base/sup/refuse.
base is defined in your supfile; by
default, base is
/usr/local/etc/cvsup,
which means that by default the refuse file is in
/usr/local/etc/cvsup/sup/refuse.The refuse file has a very simple format; it simply
contains the names of files or directories that you do not wish
to download. For example, if you cannot speak any languages other
than English and some German, and you do not feel the need to use
the German applications, you can put the following in your
refuse file:ports/chinese
ports/german
ports/japanese
ports/korean
ports/russian
ports/vietnamese
doc/es_ES.ISO8859-1
doc/ja_JP.eucJPand so forth for the other languages. Note that the name
of the repository is the first directory in the
refuse file.With this very useful feature, those users who are on
slow links or pay by the minute for their Internet connection
will be able to save valuable time as they will no longer need
to download files that they will never use. For more
information on refuse files and other neat
features of CVSup, please view its
manual page.Running CVSupYou are now ready to try an update. The command line for
doing this is quite simple:&prompt.root; cvsup supfilewhere supfile
is of course the name of the supfile you have just created.
Assuming you are running under X11, cvsup
will display a GUI window with some buttons to do the usual
things. Press the go button, and watch it
run.Since you are updating your actual
/usr/src tree in this example, you will
need to run the program as root so that
cvsup has the permissions it needs to update
your files. Having just created your configuration file, and
having never used this program before, that might
understandably make you nervous. There is an easy way to do a
trial run without touching your precious files. Just create an
empty directory somewhere convenient, and name it as an extra
argument on the command line:&prompt.root; mkdir /var/tmp/dest
&prompt.root; cvsup supfile /var/tmp/destThe directory you specify will be used as the destination
directory for all file updates.
CVSup will examine your usual files
in /usr/src, but it will not modify or
delete any of them. Any file updates will instead land in
/var/tmp/dest/usr/src.
CVSup will also leave its base
directory status files untouched when run this way. The new
versions of those files will be written into the specified
directory. As long as you have read access to
/usr/src, you do not even need to be root
to perform this kind of trial run.If you are not running X11 or if you just do not like GUIs,
you should add a couple of options to the command line when you
run cvsup:&prompt.root; cvsup -g -L 2 supfileThe tells
CVSup not to use its GUI. This is
automatic if you are not running X11, but otherwise you have to
specify it.The tells
CVSup to print out the
details of all the file updates it is doing. There are three
levels of verbosity, from to
. The default is 0, which means total
silence except for error messages.There are plenty of other options available. For a brief
list of them, type cvsup -H. For more
detailed descriptions, see the manual page.Once you are satisfied with the way updates are working, you
can arrange for regular runs of CVSup
using &man.cron.8;.
Obviously, you should not let CVSup
use its GUI when running it from &man.cron.8;.CVSup File CollectionsThe file collections available via
CVSup are organized hierarchically.
There are a few large collections, and they are divided into
smaller sub-collections. Receiving a large collection is
equivalent to receiving each of its sub-collections. The
hierarchical relationships among collections are reflected by
the use of indentation in the list below.The most commonly used collections are
src-all, and
ports-all. The other collections are used
only by small groups of people for specialized purposes, and
some mirror sites may not carry all of them.cvs-all release=cvsThe main FreeBSD CVS repository, including the
cryptography code.distrib release=cvsFiles related to the distribution and mirroring
of FreeBSD.doc-all release=cvsSources for the FreeBSD Handbook and other
documentation. This does not include files for
the FreeBSD web site.ports-all release=cvsThe FreeBSD Ports Collection.ports-archivers
release=cvsArchiving tools.ports-astro
release=cvsAstronomical ports.ports-audio
release=cvsSound support.ports-base
release=cvsMiscellaneous files at the top of
/usr/ports.ports-benchmarks
release=cvsBenchmarks.ports-biology
release=cvsBiology.ports-cad
release=cvsComputer aided design tools.ports-chinese
release=cvsChinese language support.ports-comms
release=cvsCommunication software.ports-converters
release=cvscharacter code converters.ports-databases
release=cvsDatabases.ports-deskutils
release=cvsThings that used to be on the desktop
before computers were invented.ports-devel
release=cvsDevelopment utilities.ports-editors
release=cvsEditors.ports-emulators
release=cvsEmulators for other operating
systems.ports-ftp
release=cvsFTP client and server utilities.ports-games
release=cvsGames.ports-german
release=cvsGerman language support.ports-graphics
release=cvsGraphics utilities.ports-irc
release=cvsInternet Relay Chat utilities.ports-japanese
release=cvsJapanese language support.ports-java
release=cvsJava utilities.ports-korean
release=cvsKorean language support.ports-lang
release=cvsProgramming languages.ports-mail
release=cvsMail software.ports-math
release=cvsNumerical computation software.ports-mbone
release=cvsMBone applications.ports-misc
release=cvsMiscellaneous utilities.ports-net
release=cvsNetworking software.ports-news
release=cvsUSENET news software.ports-palm
release=cvsSoftware support for 3Com Palm(tm)
series.ports-print
release=cvsPrinting software.ports-russian
release=cvsRussian language support.ports-security
release=cvsSecurity utilities.ports-shells
release=cvsCommand line shells.ports-sysutils
release=cvsSystem utilities.ports-textproc
release=cvstext processing utilities (does not
include desktop publishing).ports-vietnamese
release=cvsVietnamese language support.ports-www
release=cvsSoftware related to the World Wide
Web.ports-x11
release=cvsPorts to support the X window
system.ports-x11-clocks
release=cvsX11 clocks.ports-x11-fm
release=cvsX11 file managers.ports-x11-fonts
release=cvsX11 fonts and font utilities.ports-x11-toolkits
release=cvsX11 toolkits.ports-x11-serversX11 servers.ports-x11-wmX11 window managers.src-all release=cvsThe main FreeBSD sources, including the
cryptography code.src-base
release=cvsMiscellaneous files at the top of
/usr/src.src-bin
release=cvsUser utilities that may be needed in
single-user mode
(/usr/src/bin).src-contrib
release=cvsUtilities and libraries from outside the
FreeBSD project, used relatively unmodified
(/usr/src/contrib).src-crypto release=cvsCryptography utilities and libraries from
outside the FreeBSD project, used relatively
unmodified
(/usr/src/crypto).src-eBones release=cvsKerberos and DES
(/usr/src/eBones). Not
used in current releases of FreeBSD.src-etc
release=cvsSystem configuration files
(/usr/src/etc).src-games
release=cvsGames
(/usr/src/games).src-gnu
release=cvsUtilities covered by the GNU Public
License (/usr/src/gnu).src-include
release=cvsHeader files
(/usr/src/include).src-kerberos5
release=cvsKerberos5 security package
(/usr/src/kerberos5).src-kerberosIV
release=cvsKerberosIV security package
(/usr/src/kerberosIV).src-lib
release=cvsLibraries
(/usr/src/lib).src-libexec
release=cvsSystem programs normally executed by other
programs
(/usr/src/libexec).src-release
release=cvsFiles required to produce a FreeBSD
release
(/usr/src/release).src-secure release=cvsDES (/usr/src/secure).src-sbin
release=cvsSystem utilities for single-user mode
(/usr/src/sbin).src-share
release=cvsFiles that can be shared across multiple
systems
(/usr/src/share).src-sys
release=cvsThe kernel
(/usr/src/sys).src-sys-crypto
release=cvsKernel cryptography code
(/usr/src/sys/crypto).src-tools
release=cvsVarious tools for the maintenance of
FreeBSD
(/usr/src/tools).src-usrbin
release=cvsUser utilities
(/usr/src/usr.bin).src-usrsbin
release=cvsSystem utilities
(/usr/src/usr.sbin).www release=cvsThe sources for the FreeBSD WWW site.distrib release=selfThe CVSup server's own
configuration files. Used by CVSup
mirror sites.gnats release=currentThe GNATS bug-tracking database.mail-archive release=currentFreeBSD mailing list archive.www release=currentThe pre-processed FreeBSD WWW site files (not the
source files). Used by WWW mirror sites.For More InformationFor the CVSup FAQ and other
information about CVSup, see
The
CVSup Home Page.Most FreeBSD-related discussion of
CVSup takes place on the
&a.hackers;. New versions of the software are announced there,
as well as on the &a.announce;.Questions and bug reports should be addressed to the author
of the program at cvsup-bugs@polstra.com.CVSup SitesCVSup servers for FreeBSD are running
at the following sites:Argentinacvsup.ar.FreeBSD.org (maintainer
msagre@cactus.fi.uba.ar)Australiacvsup.au.FreeBSD.org (maintainer
dawes@xfree86.org)cvsup3.au.FreeBSD.org (maintainer
FreeBSD@admin.gil.com.au)Austriacvsup.at.FreeBSD.org (maintainer
postmaster@wu-wien.ac.at)Brazilcvsup.br.FreeBSD.org (maintainer
cvsup@cvsup.br.FreeBSD.org)cvsup2.br.FreeBSD.org (maintainer
tps@ti.sk)cvsup3.br.FreeBSD.org (maintainer
camposr@matrix.com.br)cvsup4.br.FreeBSD.org (maintainer
cvsup@tcoip.com.br)Canadacvsup.ca.FreeBSD.org (maintainer
dan@jaded.net)cvsup2.ca.FreeBSD.org (maintainer
hostmaster@ca.freebsd.org)Chinacvsup.cn.FreeBSD.org (maintainer
phj@cn.FreeBSD.org)Czech Republiccvsup.cz.FreeBSD.org (maintainer
cejkar@dcse.fee.vutbr.cz)Denmarkcvsup.dk.FreeBSD.org (maintainer
jesper@skriver.dk)Estoniacvsup.ee.FreeBSD.org (maintainer
taavi@uninet.ee)Finlandcvsup.fi.FreeBSD.org (maintainer
count@key.sms.fi)cvsup2.fi.FreeBSD.org (maintainer
count@key.sms.fi)Francecvsup.fr.FreeBSD.org (maintainer
hostmaster@fr.FreeBSD.org)cvsup2.fr.FreeBSD.org (maintainer
ftpmaint@uvsq.fr)Germanycvsup.de.FreeBSD.org (maintainer
rse@freebsd.org)cvsup1.de.FreeBSD.org (maintainer
wosch@FreeBSD.org)cvsup2.de.FreeBSD.org (maintainer
cvsup@nikoma.de)cvsup3.de.FreeBSD.org (maintainer
ag@leo.org)cvsup4.de.FreeBSD.org (maintainer
cvsup@cosmo-project.de)cvsup5.de.FreeBSD.org (maintainer
rse@freebsd.org)Greececvsup.gr.FreeBSD.org (maintainer
ftpadm@duth.gr)cvsup2.gr.FreeBSD.org (maintainer
paschos@cs.uoi.gr)Icelandcvsup.is.FreeBSD.org (maintainer
cvsup@cvsup1.is.FreeBSD.org)Irelandcvsup.ie.FreeBSD.org (maintainer
dwmalone@maths.tcd.ie),
Trinity College, Dublin.Japancvsup.jp.FreeBSD.org (maintainer
cvsupadm@jp.FreeBSD.org)cvsup2.jp.FreeBSD.org (maintainer
max@FreeBSD.org)cvsup3.jp.FreeBSD.org (maintainer
shige@cin.nihon-u.ac.jp)cvsup4.jp.FreeBSD.org (maintainer
cvsup-admin@ftp.media.kyoto-u.ac.jp)cvsup5.jp.FreeBSD.org (maintainer
cvsup@imasy.or.jp)cvsup6.jp.FreeBSD.org (maintainer
cvsupadm@jp.FreeBSD.org)Koreacvsup.kr.FreeBSD.org (maintainer
cjh@kr.FreeBSD.org)cvsup2.kr.FreeBSD.org (maintainer
holywar@mail.holywar.net)Lithuaniacvsup.lt.FreeBSD.org (maintainer
domas.mituzas@delfi.lt)New Zealandcvsup.nz.FreeBSD.org (maintainer
cvsup@langille.org)Netherlandscvsup.nl.FreeBSD.org (maintainer
xaa@xaa.iae.nl)cvsup2.nl.FreeBSD.org (maintainer
cvsup@nl.uu.net)Norwaycvsup.no.FreeBSD.org (maintainer
Per.Hove@math.ntnu.no)Polandcvsup.pl.FreeBSD.org (maintainer
Mariusz@kam.pl)Portugalcvsup.pt.FreeBSD.org (maintainer
jpedras@webvolution.net)Russiacvsup.ru.FreeBSD.org (maintainer
ache@nagual.pp.ru)cvsup2.ru.FreeBSD.org (maintainer
dv@dv.ru)cvsup3.ru.FreeBSD.org (maintainer
fjoe@iclub.nsu.ru)cvsup4.ru.FreeBSD.org (maintainer
zhecka@klondike.ru)cvsup5.ru.FreeBSD.org (maintainer
maxim@macomnet.ru)cvsup6.ru.FreeBSD.org (maintainer
pvr@corbina.net)Slovak Republiccvsup.sk.FreeBSD.org (maintainer
tps@tps.sk)cvsup2.sk.FreeBSD.org (maintainer
tps@tps.sk)Sloveniacvsup.si.FreeBSD.org (maintainer
blaz@si.FreeBSD.org)South Africacvsup.za.FreeBSD.org (maintainer
markm@FreeBSD.org)cvsup2.za.FreeBSD.org (maintainer
markm@FreeBSD.org)Spaincvsup.es.FreeBSD.org (maintainer
jesusr@FreeBSD.org)cvsup2.es.FreeBSD.org (maintainer
jesusr@FreeBSD.org)cvsup3.es.FreeBSD.org (maintainer
jose@we.lc.ehu.es)Swedencvsup.se.FreeBSD.org (maintainer
pantzer@ludd.luth.se)cvsup2.se.FreeBSD.org (maintainer
cvsup@dataphone.net)Taiwancvsup.tw.FreeBSD.org (maintainer
jdli@freebsd.csie.nctu.edu.tw)cvsup2.tw.FreeBSD.org (maintainer
ycheng@sinica.edu.tw)cvsup3.tw.FreeBSD.org (maintainer
foxfair@FreeBSD.org)Ukrainecvsup2.ua.FreeBSD.org (maintainer
freebsd-mnt@lucky.net)cvsup3.ua.FreeBSD.org (maintainer
ftpmaster@ukr.net), Kievcvsup4.ua.FreeBSD.org (maintainer
phantom@cris.net)United Kingdomcvsup.uk.FreeBSD.org (maintainer
ftp-admin@plig.net)cvsup2.uk.FreeBSD.org (maintainer
brian@FreeBSD.org)cvsup3.uk.FreeBSD.org (maintainer
ben.hughes@uk.easynet.net)USAcvsup1.FreeBSD.org (maintainer
cwt@networks.cwu.edu), Washington
statecvsup2.FreeBSD.org (maintainer
jdp@FreeBSD.org), Californiacvsup3.FreeBSD.org (maintainer
wollman@FreeBSD.org), Massachusettscvsup5.FreeBSD.org (maintainer
mjr@blackened.com), Arizonacvsup6.FreeBSD.org (maintainer
cvsup@cvsup.adelphiacom.net), Illinoiscvsup7.FreeBSD.org (maintainer
jdp@FreeBSD.org), Washington statecvsup8.FreeBSD.org (maintainer
hostmaster@bigmirror.com), Washington
statecvsup9.FreeBSD.org (maintainer
qbsd@uswest.net), Minnesotacvsup10.FreeBSD.org (maintainer
jdp@FreeBSD.org), Californiacvsup11.FreeBSD.org (maintainer
cvsup@research.uu.net), Virginiacvsup12.FreeBSD.org (maintainer
will@FreeBSD.org), Indianacvsup13.FreeBSD.org (maintainer
dima@valueclick.com), Californiacvsup14.FreeBSD.org (maintainer
freebsd-cvsup@mfnx.net), Californiacvsup15.FreeBSD.org (maintainer
cvsup@math.uic.edu), Illinoiscvsup16.FreeBSD.org (maintainer
pth3k@virginia.edu), Virginiacvsup17.FreeBSD.org (maintainer
cvsup@mirrortree.com), Washington stateCVS TagsWhen obtaining or updating sources from
cvs and
CVSup a revision tag (reference to a
date in time) must be specified. The following tags are
available, each specifying different branches of FreeBSD at
different points of time:The ports tree does not have any tag associated with it,
it is always CURRENT.The most common tags are:HEADSymbolic name for the main line, or FreeBSD-CURRENT.
Also the default when no revision is specified.In CVSup, this tag is represented
by a . (not punctuation, but a literal
. character).In CVS, this is the default when no revision tag is
specified. It is usually not
a good idea to checkout or update to CURRENT sources
on a STABLE machine, unless that is your intent.RELENG_4The line of development for FreeBSD-4.X, also known
as FreeBSD-STABLE.RELENG_4_4The release branch for FreeBSD-4.4, used only
for security advisories and other seriously critical fixes.RELENG_4_3The release branch for FreeBSD-4.3, used only
for security advisories and other seriously critical fixes.RELENG_3The line of development for FreeBSD-3.X, also known
as 3.X-STABLE.RELENG_2_2The line of development for FreeBSD-2.2.X, also known
as 2.2-STABLE. This branch is mostly obsolete.Other revision tags that are available include:RELENG_4_4_0_RELEASEFreeBSD 4.4.RELENG_4_3_0_RELEASEFreeBSD 4.3.RELENG_4_2_0_RELEASEFreeBSD 4.2.RELENG_4_1_1_RELEASEFreeBSD 4.1.1.RELENG_4_1_0_RELEASEFreeBSD 4.1.RELENG_4_0_0_RELEASEFreeBSD 4.0.RELENG_3_5_0_RELEASEFreeBSD-3.5.RELENG_3_4_0_RELEASEFreeBSD-3.4.RELENG_3_3_0_RELEASEFreeBSD-3.3.RELENG_3_2_0_RELEASEFreeBSD-3.2.RELENG_3_1_0_RELEASEFreeBSD-3.1.RELENG_3_0_0_RELEASEFreeBSD-3.0.RELENG_2_2_8_RELEASEFreeBSD-2.2.8.RELENG_2_2_7_RELEASEFreeBSD-2.2.7.RELENG_2_2_6_RELEASEFreeBSD-2.2.6.RELENG_2_2_5_RELEASEFreeBSD-2.2.5.RELENG_2_2_2_RELEASEFreeBSD-2.2.2.RELENG_2_2_1_RELEASEFreeBSD-2.2.1.RELENG_2_2_0_RELEASEFreeBSD-2.2.0.AFS SitesAFS servers for FreeBSD are running at the following sites;SwedenThe path to the files are:
/afs/stacken.kth.se/ftp/pub/FreeBSD/stacken.kth.se # Stacken Computer Club, KTH, Sweden
130.237.234.43 #hot.stacken.kth.se
130.237.237.230 #fishburger.stacken.kth.se
130.237.234.3 #milko.stacken.kth.seMaintainer ftp@stacken.kth.se
diff --git a/en_US.ISO8859-1/books/porters-handbook/book.sgml b/en_US.ISO8859-1/books/porters-handbook/book.sgml
index f4e50b6aea..0d0f2c27cc 100644
--- a/en_US.ISO8859-1/books/porters-handbook/book.sgml
+++ b/en_US.ISO8859-1/books/porters-handbook/book.sgml
@@ -1,4564 +1,4564 @@
%man;
%bookinfo;
%authors;
%mailing-lists;
]>
FreeBSD Porter's HandbookThe FreeBSD Documentation ProjectApril 20002000The FreeBSD Documentation
Project
&bookinfo.legalnotice;
Making a port yourselfSo, now you are interested in making your own port or
upgrading an existing one? Great!What follows are some guidelines for creating a new port for
FreeBSD. If you want to upgrade an existing port, you should
read this and then read .When this document is not sufficiently detailed, you should
refer to /usr/ports/Mk/bsd.port.mk, which
all port Makefiles include. Even if you do not hack Makefiles
daily, it is well commented, and you will still gain much
knowledge from it. Additionally, you may send specific questions
to the &a.ports;.Only a fraction of the variables
(VAR) that can be
overridden are mentioned in this document. Most (if not all)
are documented at the start of bsd.port.mk.
This file uses a non-standard tab setting.
Emacs and
Vim should recognize the setting on
loading the file. Both vi and
ex can be set to use the correct value by
typing :set tabstop=4 once the file has been
loaded.Quick PortingThis section tells you how to do a quick port. In many cases, it
is not enough, but we will see.First, get the original tarball and put it into
DISTDIR, which defaults to
/usr/ports/distfiles.The following assumes that the software compiled out-of-the-box,
i.e., there was absolutely no change required for the port to work
on your FreeBSD box. If you needed to change something, you will
have to refer to the next section too.Writing the MakefileThe minimal Makefile would look something
like this:# New ports collection makefile for: oneko
# Date created: 5 December 1994
# Whom: asami
#
# $FreeBSD$
#
PORTNAME= oneko
PORTVERSION= 1.1b
CATEGORIES= games
MASTER_SITES= ftp://ftp.cs.columbia.edu/archives/X11R5/contrib/
MAINTAINER= asami@FreeBSD.org
MAN1= oneko.1
MANCOMPRESSED= yes
USE_IMAKE= yes
.include <bsd.port.mk>See if you can figure it out. Do not worry about the contents
of the $FreeBSD$ line, it will be
filled in automatically by CVS when the port is imported to our main
ports tree. You can find a more detailed example in the sample Makefile section.Writing the description filesThere are three description files that are required for
any port, whether they actually package or not. They are
pkg-comment,
pkg-descr, and
pkg-plist, and their
pkg- prefix distinguishes them from
other files.pkg-commentThis is the one-line description of the port.
Please do not include the package name (or
version number of the software) in the comment. The comment
should begin with a capital, and end without a period. Here
is an example:A cat chasing a mouse all over the screenpkg-descrThis is a longer description of the port. One to a few
paragraphs concisely explaining what the port does is
sufficient.This is not a manual or an in-depth
description on how to use or compile the port! Please
be careful if you are copying from the
README or manpage; too often
they are not a concise description of the port or are in an
awkward format (e.g., manpages have justified spacing). If the
ported software has an official WWW homepage, you should list it
here. Prefix one of the websites with
WWW: so that automated tools will work
correctly.It is recommended that you sign your name at the end of this
file, as in:This is a port of oneko, in which a cat chases a poor mouse all over
the screen.
:
(etc.)
WWW: http://www.oneko.org/
- Satoshi
asami@cs.berkeley.edupkg-plistThis file lists all the files installed by the port. It is
- also called the “packing list” because the package is
+ also called the packing list because the package is
generated by packing the files listed here. The pathnames are
relative to the installation prefix (usually
/usr/local or
/usr/X11R6). If you are using the
MANn variables (as
you should be), do not list any manpages here.Here is a small example:bin/oneko
lib/X11/app-defaults/Oneko
lib/X11/oneko/cat1.xpm
lib/X11/oneko/cat2.xpm
lib/X11/oneko/mouse.xpm
@dirrm lib/X11/onekoRefer to the &man.pkg.create.1; man page for details on the
packing list.You should list all the files, but not the name directories,
in the list. Also, if the port creates directories for itself
during installation, make sure to add @dirrm
lines as necessary to remove them when the port is
deleted.It is recommended that you keep all the filenames in this
file sorted alphabetically. It will make verifying the changes
when you upgrade the port much easier.Creating a packing list manually can be a very tedious
task. If the port installs a large numbers of files, creating the packing list
automatically might save time.Creating the checksum fileJust type make makesum. The ports make rules
will automatically generate the file
distinfo.Testing the portYou should make sure that the port rules do exactly what you
want them to do, including packaging up the port. These are the
important points you need to verify.pkg-plist does not contain anything not
installed by your portpkg-plist contains everything that is
installed by your portYour port can be installed multiple times using the
reinstall targetYour port cleans up
after itself upon deinstallRecommended test orderingmake installmake packagemake deinstallpkg_add package-namemake deinstallmake reinstallmake packageMake sure that there are not any warnings issued in any of the
package and
deinstall stages. After step 3, check to
see if all the new directories are correctly deleted. Also, try
using the software after step 4, to ensure that it works correctly
when installed from a package.Checking your port with portlintPlease use portlint to see if your port
conforms to our guidelines. The portlint program
is part of the ports collection. In particular, you may want to
check if the Makefile is in
the right shape and the package is named
appropriately.Submitting the portFirst, make sure you have read the DOs and DON'Ts section.Now that you are happy with your port, the only thing remaining
is to put it in the main FreeBSD ports tree and make everybody else
happy about it too. We do not need your work
directory or the pkgname.tgz package, so delete
them now. Next, simply include the output of shar `find
port_dir` in a bug report and send it with the
&man.send-pr.1; program (see Bug
Reports and General Commentary for more information about
&man.send-pr.1;. If the uncompressed port is larger than 20KB,
you should compress it into a tarfile and use &man.uuencode.1;
before including it in the bug report (uuencoded tarfiles are
acceptable even if the bug report is smaller than 20KB but are not
preferred). Be sure to classify the bug report as category
ports and class
change-request (Do not mark the report
confidential!).
Also add a short description of the program you ported
to the Description field of the PR and
the shar or uuencoded tarfile to the
Fix field. The latter one helps the committers
a lot, who use scripts for the ports-work.One more time, do not include the original source
distfile, the work directory, or the package
you built with make package.In the past, we asked you to upload new port submissions in
our FTP site (ftp.FreeBSD.org). This
is no longer recommended as read access is turned off on the
incoming/ directory of that site due to the
large amount of pirated software showing up there.We will look at your port, get back to you if necessary, and put
it in the tree. Your name will also appear in the list of
- “Additional FreeBSD contributors” in the FreeBSD
+ Additional FreeBSD contributors in the FreeBSD
Handbook and other files. Isn't that great?!? :-)You can make our work a lot easier, if you use a good
description in the synopsis of the problem report.
We prefer something like
- “New port: <short description of the port>” for
+ New port: <short description of the port> for
new ports and
- “Update port: <category>/<port> <short description
- of the update>” for port updates.
+ Update port: <category>/<port> <short description
+ of the update> for port updates.
If you stick to this scheme, the chance that one takes a look at
your PR soon is much bigger.Slow PortingOk, so it was not that simple, and the port required some
modifications to get it to work. In this section, we will explain,
step by step, how to modify it to get it to work with the ports
paradigm.How things workFirst, this is the sequence of events which occurs when the user
first types make in your port's directory.
You may find that having bsd.port.mk in another
window while you read this really helps to understand it.But do not worry if you do not really understand what
bsd.port.mk is doing, not many people do...
:->The fetch target is run. The
fetch target is responsible for making
sure that the tarball exists locally in
DISTDIR. If fetch
cannot find the required files in DISTDIR it
will look up the URL MASTER_SITES, which is
set in the Makefile, as well as our main FTP site at ftp://ftp.FreeBSD.org/pub/FreeBSD/ports/distfiles/,
where we put sanctioned distfiles as backup. It will then
attempt to fetch the named distribution file with
FETCH, assuming that the requesting site has
direct access to the Internet. If that succeeds, it will save
the file in DISTDIR for future use and
proceed.The extract target is run. It
looks for your port's distribution file (typically a gzip'd
tarball) in DISTDIR and unpacks it into a
temporary subdirectory specified by WRKDIR
(defaults to work).The patch target is run. First,
any patches defined in PATCHFILES are
applied. Second, if any patch files named
patch-* are found in
PATCHDIR (defaults to the
files subdirectory), they are applied at
this time in alphabetical order.The configure target is run. This
can do any one of many different things.If it exists, scripts/configure is
run.If HAS_CONFIGURE or
GNU_CONFIGURE is set,
WRKSRC/configure is
run.If USE_IMAKE is set,
XMKMF (default: xmkmf
-a) is run.The build target is run. This is
responsible for descending into the port's private working
directory (WRKSRC) and building it. If
USE_GMAKE is set, GNU make
will be used, otherwise the system make will
be used.The above are the default actions. In addition, you can define
targets
pre-something or
post-something,
or put scripts with those names, in the scripts
subdirectory, and they will be run before or after the default
actions are done.For example, if you have a post-extract
target defined in your Makefile, and a file
pre-build in the scripts
subdirectory, the post-extract target will
be called after the regular extraction actions, and the
pre-build script will be executed before the
default build rules are done. It is recommended that you use
Makefile targets if the actions are simple
enough, because it will be easier for someone to figure out what
kind of non-default action the port requires.The default actions are done by the
bsd.port.mk targets
do-something.
For example, the commands to extract a port are in the target
do-extract. If you are not happy with the
default target, you can fix it by redefining the
do-something
target in your Makefile.
- The “main” targets (e.g.,
+ The main targets (e.g.,
extract,
configure, etc.) do nothing more than
make sure all the stages up to that one are completed and call
the real targets or scripts, and they are not intended to be
changed. If you want to fix the extraction, fix
do-extract, but never ever touch
extract!Now that you understand what goes on when the user types
make, let us go through the recommended steps to
create the perfect port.Getting the original sourcesGet the original sources (normally) as a compressed tarball
(foo.tar.gz or
foo.tar.Z) and copy
it into DISTDIR. Always use
mainstream sources when and where you
can.If you cannot find a FTP/http site that is well-connected to the
net, or can only find sites that have irritatingly non-standard
formats, you might want to put a copy on a reliable FTP or http
server that you control (e.g., your home page). Make sure you set
MASTER_SITES to reflect your choice.If you cannot find somewhere convenient and reliable to put the
distfile
- we can “house” it ourselves
+ we can house it ourselves
on ftp.FreeBSD.org.
The distfile must be placed into
~/public_distfiles/ of someone's
freefall account.
Ask the person who commits your port to do this.
This person will also set MASTER_SITES to
MASTER_SITE_LOCAL and
MASTER_SITE_SUBDIR to their
freefall username.If your port's distfile changes all the time for no good reason,
consider putting the distfile in your home page and listing it as
the first MASTER_SITES. This will prevent users
from getting checksum mismatch errors, and
also reduce the workload of maintainers of our FTP site. Also, if
there is only one master site for the port, it is recommended that
you house a backup at your site and list it as the second
MASTER_SITES.If your port requires some additional `patches' that are
available on the Internet, fetch them too and put them in
DISTDIR. Do not worry if they come from a site
other than where you got the main source tarball, we have a way to
handle these situations (see the description of PATCHFILES below).Modifying the portUnpack a copy of the tarball in a private directory and make
whatever changes are necessary to get the port to compile properly
under the current version of FreeBSD. Keep careful
track of everything you do, as you will be automating
the process shortly. Everything, including the deletion, addition,
or modification of files should be doable using an automated script
or patch file when your port is finished.If your port requires significant user interaction/customization
to compile or install, you should take a look at one of Larry Wall's
classic Configure scripts and perhaps do
something similar yourself. The goal of the new ports collection is
- to make each port as “plug-and-play” as possible for the
+ to make each port as plug-and-play as possible for the
end-user while using a minimum of disk space.Unless explicitly stated, patch files, scripts, and other
files you have created and contributed to the FreeBSD ports
collection are assumed to be covered by the standard BSD copyright
conditions.PatchingIn the preparation of the port, files that have been added or
changed can be picked up with a recursive diff for later feeding to
patch. Each set of patches you wish to apply should be collected
into a file named
patch-* where
* denotes the sequence in which the
patches will be applied — these are done in
alphabetical order, thus aa
first, ab second and so on. If you wish,
you can use names that indicate the pathnames of the files that
are patched, such as patch-Imakefile or
patch-src-config.h. These files should
be stored in PATCHDIR, from where they will be
automatically applied. All patches should be relative to
WRKSRC (generally the directory your port's
tarball unpacks itself into, that being where the build is done).
To make fixes and upgrades easier, you should avoid having more than
one patch fix the same file (e.g., patch-aa and
patch-ab both changing
WRKSRC/foobar.c).ConfiguringInclude any additional customization commands in your
configure script and save it in the
scripts subdirectory. As mentioned above, you
can also do this with Makefile targets and/or
scripts with the name pre-configure or
post-configure.Handling user inputIf your port requires user input to build, configure, or install,
then set IS_INTERACTIVE in your Makefile. This
- will allow “overnight builds” to skip your port if the
+ will allow overnight builds to skip your port if the
user sets the variable BATCH in his environment (and
if the user sets the variable INTERACTIVE, then
only those ports requiring interaction are
built).It is also recommended that if there are reasonable default
answers to the questions, you check the
PACKAGE_BUILDING variable and turn off the
interactive script when it is set. This will allow us to build the
packages for CDROMs and FTP.Configuring the MakefileConfiguring the Makefile is pretty simple, and again we suggest
that you look at existing examples before starting. Also, there is a
sample Makefile in this
handbook, so take a look and please follow the ordering of variables
and sections in that template to make your port easier for others to
read.Now, consider the following problems in sequence as you design
your new Makefile:The original sourceDoes it live in DISTDIR as a standard
gzip'd tarball named something like
foozolix-1.2.tar.gz? If so, you can go on
to the next step. If not, you should look at overriding any of
the DISTNAME, EXTRACT_CMD,
EXTRACT_BEFORE_ARGS,
EXTRACT_AFTER_ARGS,
EXTRACT_SUFX, or DISTFILES
variables, depending on how alien a format your port's
distribution file is. (The most common case is
EXTRACT_SUFX=.tar.Z, when the tarball is
condensed by regular compress, not
gzip.)In the worst case, you can simply create your own
do-extract target to override the
default, though this should be rarely, if ever,
necessary.PORTNAME and PORTVERSIONYou should set PORTNAME to the
base name of your port, and PORTVERSION
to the version number of the port.PORTREVISION and
PORTEPOCHPORTREVISIONThe PORTREVISION variable is a
monotonically increasing value which is reset to 0 with
every increase of PORTVERSION (i.e.
every time a new official vendor release is made), and
appended to the package name if non-zero.
PORTREVISION is increased each time a
change is made to the FreeBSD port which significantly
affects the content or stucture of the derived
package.Examples of when PORTREVISION should be bumped:Addition of patches to correct security
vulnerabilities, bugs, or to add new functionality to
the FreeBSD port.Changes to the port makefile to enable or disable
compile-time options in the package.Changes in the packing list or the install-time
behaviour of the package (e.g. change to a script
which generates initial data for the package, like ssh
host keys).Version bump of a port's shared library dependency
(in this case, someone trying to install the old
package after installing a newer version of the
dependency will fail since it will look for the old
libfoo.x instead of libfoo.(x+1)).Silent changes to the port distfile which have
significant functional differences, i.e. changes to
the distfile requiring a correction to
distinfo with no corresponding change to
PORTVERSION, where a diff
-ru of the old and new versions shows
non-trivial changes to the code.Examples of changes which do not require a
PORTREVISION bump:Style changes to the port skeleton with no
functional change to what appears in the resulting
package.Changes to MASTER_SITES or
other functional changes to the port which do not
effect the resulting package.Trivial patches to the distfile such as correction
of typos, which are not important enough that users of
the package should go to the trouble of
upgrading.Build fixes which cause a package to become
compilable where it was previously failing (as long as
the changes do not introduce any functional change on
any other platforms on which the port did previously
build). Since PORTREVISION reflects
the content of the package, if no package was
previously buildable then there is no need to increase
PORTREVISION to mark a
change.A rule of thumb is to ask yourself whether a change
committed to a port is something which someone, somewhere,
would benefit from having (either because of an
enhancement, fix, or by virtue that the new package will
actually work for them). If yes, the
PORTREVISION should be bumped so that
automated tools (e.g. pkg_version)
will highlight the fact that a new package is
available.PORTEPOCHFrom time to time a software vendor or FreeBSD porter
will do something silly and release a version of their
software which is actually numerically less than the
previous version. An example of this is a port which goes
from foo-20000801 to foo-1.0 (the former will be
incorrectly treated as a newer version since 20000801 is a
numerically greater value than 1).In situations such as this, the
PORTEPOCH version should be increased.
If PORTEPOCH is nonzero it is appended
to the package name as described in section 0 above.
PORTEPOCH is never decreased or reset
to zero, because that would cause comparison to a package
from an earlier epoch to fail (i.e. the package would not
be detected as out of date): the new version number (e.g.
1.0,1 in the above example) is still
numerically less than the previous version (20000801), but
the ,1 suffix is treated specially by
automated tools and found to be greater than the implied
suffix ",0" on the earlier package.It is expected that PORTEPOCH will
not be used for the majority of ports, and that sensible
use of PORTVERSION can often pre-empt
it becoming necessary if a future release of the software
should change the version structure. However, care is
needed by FreeBSD porters when a vendor release is made
without an official version number - such as a code
"snapshot" release. The temptation is to label the
release with the release date, which will cause problems
as in the example above when a new "official" release is
made.For example, if a snapshot release is made on the date
20000917, and the previous version of the software was
version 1.2, the snapshot release should be given a
PORTVERSION of 1.2.20000917 or similar,
not 20000917, so that the succeeding release, say 1.3, is
still a numerically greater value.Example of PORTREVISION and
PORTEPOCH usageThe gtkmumble port, version 0.10, is committed to the
ports collection.PORTNAME= gtkmumble
PORTVERSION= 0.10PKGNAME becomes
gtkmumble-0.10.A security hole is discovered which requires a local
FreeBSD patch. PORTREVISION is bumped
accordingly.PORTNAME= gtkmumble
PORTVERSIOn= 0.10
PORTREVISION= 1PKGNAME becomes
gtkmumble-0.10_1A new version is released by the vendor, numbered 0.2
(it turns out the author actually intended
0.10 to actually mean
0.1.0, not what comes after
0.9 - oops, too late now). Since the new minor
version 2 is numerically less than the
previous version 10 the
PORTEPOCH must be bumped to manually
force the new package to be detected as "newer". Since it
is a new vendor release of the code,
PORTREVISION is reset to 0 (or removed
from the makefile).PORTNAME= gtkmumble
PORTVERSION= 0.2
PORTEPOCH= 1PKGNAME becomes
gtkmumble-0.2,1The next release is 0.3. Since
PORTEPOCH never decreases, the version
variables are now:PORTNAME= gtkmumble
PORTVERSION= 0.3
PORTEPOCH= 1PKGNAME becomes
gtkmumble-0.3,1If PORTEPOCH were reset
to 0 with this upgrade, someone who had
installed the gtkmumble-0.10_1 package would not detect
the gtkmumble-0.3 package as newer, since
3 is still numerically less than
10.PKGNAMEPREFIX and PKGNAMESUFFIXTwo optional variables, PKGNAMEPREFIX and
PKGNAMESUFFIX, are combined with
PORTNAME and
PORTVERSION to
form PKGNAME as
${PKGNAMEPREFIX}${PORTNAME}${PKGNAMESUFFIX}-${PORTVERSION}.
Make sure this conforms to our guidelines for a good package
name. In particular, you are not allowed to use a
hyphen (-) in
PORTVERSION. Also, if the package name
has the language- or the
compiled.specifics part, use
PKGNAMEPREFIX and
PKGNAMESUFFIX, respectively. Do not make
them part of PORTNAME.DISTNAMEDISTNAME is the name of the port as
called by the authors of the software.
DISTNAME defaults to
${PORTNAME}-${PORTVERSION}, so override it if necessary.
DISTNAME is only used in two places.
First, the distribution file list
(DISTFILES) defaults to
${DISTNAME}${EXTRACT_SUFX}.
Second, the distribution file is expected to extract into a
subdirectory named WRKSRC, which defaults
to work/${DISTNAME}.PKGNAMEPREFIX and
PKGNAMESUFFIX do not affect
DISTNAME. Also note that when
WRKSRC is equal to
work/${PORTNAME}-${PORTVERSION}
while the original source archive is named something other than
${PORTNAME}-${PORTVERSION}${EXTRACT_SUFX},
you should probably leave DISTNAME
alone— you are better off defining
DISTFILES than having to set both
DISTNAME and WRKSRC
(and possibly EXTRACT_SUFX).CATEGORIESWhen a package is created, it is put under
/usr/ports/packages/All and links are made from
one or more subdirectories of
/usr/ports/packages. The names of these
subdirectories are specified by the variable
CATEGORIES. It is intended to make life easier
for the user when he is wading through the pile of packages on the
FTP site or the CDROM. Please take a look at the existing categories and pick the ones
that are suitable for your port.This list also determines where in the ports tree the port is
imported. If you put more than one category here, it is assumed
that the port files will be put in the subdirectory with the name in
the first category. See the categories section for more
discussion about how to pick the right categories.If your port truly belongs to something that is different from
all the existing ones, you can even create a new category name. In
that case, please send mail to the &a.ports; to propose a new
category.MASTER_SITESRecord the directory part of the FTP/http-URL pointing at the
original tarball in MASTER_SITES. Do not forget
the trailing slash (/)!The make macros will try to use this
specification for grabbing the distribution file with
FETCH if they cannot find it already on the
system.It is recommended that you put multiple sites on this list,
preferably from different continents. This will safeguard against
wide-area network problems, and we are even planning to add support
for automatically determining the closest master site and fetching
from there!If the original tarball is part of one of the popular
archives such as X-contrib, GNU, or Perl CPAN, you may be able
refer to those sites in an easy compact form using
MASTER_SITE_*
(e.g., MASTER_SITE_XCONTRIB and
MASTER_SITE_PERL_GNU). Simply set
MASTER_SITES to one of these variables and
MASTER_SITE_SUBDIR to the path within the
archive. Here is an example:MASTER_SITES= ${MASTER_SITE_XCONTRIB}
MASTER_SITE_SUBDIR= applicationsThese variables are defined in
/usr/ports/Mk/bsd.sites.mk. There are
new archives added all the time, so make sure to check the
latest version of this file before submitting a port.The user can also set the MASTER_SITE_*
variables in /etc/make.conf to override our
choices, and use their favorite mirrors of these popular archives
instead.PATCHFILESIf your port requires some additional patches that are available
by FTP or http, set PATCHFILES to the names of
the files and PATCH_SITES to the URL of the
directory that contains them (the format is the same as
MASTER_SITES).If the patch is not relative to the top of the source tree
(i.e., WRKSRC) because it contains some extra
pathnames, set PATCH_DIST_STRIP accordingly. For
instance, if all the pathnames in the patch have an extra
foozolix-1.0/ in front of the filenames, then set
PATCH_DIST_STRIP=-p1.Do not worry if the patches are compressed; they will be
decompressed automatically if the filenames end with
.gz or .Z.If the patch is distributed with some other files, such as
documentation, in a gzip'd tarball, you cannot just use
PATCHFILES. If that is the case, add the name
and the location of the patch tarball to
DISTFILES and MASTER_SITES.
Then, use the EXTRA_PATCHES variable to
point to those files and bsd.port.mk
will automatically apply them for you. In particular, do
not copy patch files into the
PATCHDIR directory—that directory may
not be writable.Note that the tarball will have been extracted alongside the
regular source by then, so there is no need to explicitly extract
it if it is a regular gzip'd or compress'd tarball. If you do the
latter, take extra care not to overwrite something that already
exists in that directory. Also, do not forget to add a command to
remove the copied patch in the pre-clean
target.MAINTAINERSet your mail-address here. Please. :-)For a detailed description of the responsibilities of maintainers,
refer to the MAINTAINER on
Makefiles section.DependenciesMany ports depend on other ports. There are five variables that
you can use to ensure that all the required bits will be on the
user's machine. There are also some pre-supported dependency
variables for common cases, plus a few more to control the behaviour
of dependencies.LIB_DEPENDSThis variable specifies the shared libraries this port depends
on. It is a list of
lib:dir:target
tuples where lib is the name of the
shared library, dir is the
directory in which to find it in case it is not available, and
target is the target to call in that
directory. For example, LIB_DEPENDS=
jpeg.9:${PORTSDIR}/graphics/jpeg:install
will check for a shared jpeg library with major version 9, and
descend into the graphics/jpeg subdirectory
of your ports tree to build and install it if it is not found.
The target part can be omitted if it is
equal to DEPENDS_TARGET (which defaults to
install).The lib part is an argument given
to ldconfig -r | grep -wF. There shall be no
regular expressions in this variable.The dependency is checked twice, once from within the
extract target and then from within the
install target. Also, the name of the
dependency is put into the package so that
pkg_add will automatically install it if it is
not on the user's system.RUN_DEPENDSThis variable specifies executables or files this port depends
on during run-time. It is a list of
path:dir:target
tuples where path is the name of the
executable or file, dir is the
directory in which to find it in case it is not available, and
target is the target to call in that
directory. If path starts with a slash
(/), it is treated as a file and its existence
is tested with test -e; otherwise, it is
assumed to be an executable, and which -s is
used to determine if the program exists in the user's search
path.For example,RUN_DEPENDS= ${PREFIX}/etc/innd:${PORTSDIR}/news/inn \
wish8.0:${PORTSDIR}/x11-toolkits/tk80will check if the file or directory
/usr/local/etc/innd exists, and build and
install it from the news/inn subdirectory of
the ports tree if it is not found. It will also see if an
executable called wish8.0 is in your search
path, and descend into the x11-toolkits/tk80
subdirectory of your ports tree to build and install it if it is
not found.In this case, innd is actually an
executable; if an executable is in a place that is not expected
to be in a normal user's search path, you should use the full
pathname.The dependency is checked from within the
install target. Also, the name of the
dependency is put in to the package so that
pkg_add will automatically install it if it is
not on the user's system. The target
part can be omitted if it is the same as
DEPENDS_TARGET.BUILD_DEPENDSThis variable specifies executables or files this port
requires to build. Like RUN_DEPENDS, it is a
list of
path:dir:target
tuples. For example, BUILD_DEPENDS=
unzip:${PORTSDIR}/archivers/unzip will check
for an executable called unzip, and descend
into the archivers/unzip subdirectory of your
ports tree to build and install it if it is not found.
- “build” here means everything from extraction to
+ build here means everything from extraction to
compilation. The dependency is checked from within the
extract target. The
target part can be omitted if it is
the same as DEPENDS_TARGETFETCH_DEPENDSThis variable specifies executables or files this port
requires to fetch. Like the previous two, it is a list of
path:dir:target
tuples. For example, FETCH_DEPENDS=
ncftp2:${PORTSDIR}/net/ncftp2 will check for an
executable called ncftp2, and descend into the
net/ncftp2 subdirectory of your ports tree to
build and install it if it is not found.The dependency is checked from within the
fetch target. The
target part can be omitted if it is the
same as DEPENDS_TARGET.DEPENDSIf there is a dependency that does not fall into either of the
above four categories, or your port requires having the source of
the other port extracted in addition to having it installed,
then use this variable. This is a list of
dir:target,
as there is nothing to check, unlike the previous four. The
target part can be omitted if it is the
same as DEPENDS_TARGET.Common dependency variablesDefine USE_XLIB=yes if your port requires
the X Window System to be installed (it is implied by
USE_IMAKE). Define
USE_GMAKE=yes if your port requires GNU
make instead of BSD make.
Define USE_AUTOCONF=yes if your port requires
GNU autoconf to be run. Define USE_QT=yes if
your port uses the latest qt toolkit. Use
USE_PERL5=yes if your port requires version 5
of the perl language. (The last is especially important since
some versions of FreeBSD have perl5 as part of the base system
while others do not.)Notes on dependenciesAs mentioned above, the default target to call when a
dependency is required is DEPENDS_TARGET.
It defaults to install. This is a user
variable; it is never defined in a port's
Makefile. If your port needs a special way
to handle a dependency, use the :target part of
the *_DEPENDS variables instead of redefining
DEPENDS_TARGET.When you type make clean, its dependencies
are automatically cleaned too. If you do not wish this to happen,
define the variable NOCLEANDEPENDS in your
environment.To depend on another port unconditionally, use the
variable ${NONEXISTENT} as the first field
of BUILD_DEPENDS or
RUN_DEPENDS. Use this only when you need to
the to get to the source of the other port. You can often save
compilation time by specifying the target too. For
instance
BUILD_DEPENDS= ${NONEXISTENT}:${PORTSDIR}/graphics/jpeg:extract
will always descend to the JPEG port and extract it.Do not use DEPENDS unless there is no other
way the behaviour you want can be accomplished. It will cause the
other port to always be built (and installed, by default), and the
dependency will go into the packages as well. If this is really
what you need, you should probably write it as
BUILD_DEPENDS and
RUN_DEPENDS instead—at least the
intention will be clear.Optional dependenciesSome large applications can be built in a number of
configurations, adding functionality if one of a number of
libraries or applications is available. Since not all users
want those libraries or applications, the ports system
provides hooks that the port author can use to decide which
configuration should be built. Supporting these properly will
make uses happy, and effectively provide 2 or more ports for the
price of one.The easiest of these to use is
WITHOUT_X11. If the port can be built both
with and without X support, then it should normally be built
with X support. If WITHOUT_X11 is defined,
then the version that does not have X support should be
built.Various parts of GNOME have such knobs, though they are
slightly more difficult to use. The variables to use in the
Makefile are WANT_*
and HAVE_*. If the application can be
built both with or without one of the dependencies listed
below, then the Makefile should set
WANT_PKG, and should build the version that
uses PKG if HAVE_PKG
is defined.The WANT_* variables currently
supported this way are WANT_GLIB,
WANT_GTK, WANT_ESOUND,
WANT_IMLIB, and
WANT_GNOME.Building mechanismsIf your package uses GNU make, set
USE_GMAKE=yes. If your package uses
configure, set
HAS_CONFIGURE=yes. If your package uses GNU
configure, set
GNU_CONFIGURE=yes (this implies
HAS_CONFIGURE). If you want to give some extra
arguments to configure (the default argument list
--prefix=${PREFIX} for GNU
configure and empty for non-GNU
configure), set those extra arguments in
CONFIGURE_ARGS. If your package uses GNU
autoconf, set
USE_AUTOCONF=yes. This implies
GNU_CONFIGURE, and will cause
autoconf to be run before
configure.If your package is an X application that creates
Makefiles from Imakefiles
using imake, then set
USE_IMAKE=yes. This will cause the configure
stage to automatically do an xmkmf -a. If the
flag is a problem for your port, set
XMKMF=xmkmf. If the port uses
imake but does not understand the
install.man target,
NO_INSTALL_MANPAGES=yes should be set. In
addition, the author of the original port should be shot. :->If your port's source Makefile has
something else than all as the main build
target, set ALL_TARGET accordingly. Same goes
for install and
INSTALL_TARGET.Special considerationsThere are some more things you have to take into account when you
create a port. This section explains the most common of those.Shared LibrariesIf your port installs one or more shared libraries, define a
INSTALLS_SHLIB make variable, which will instruct
a bsd.port.mk to run
${LDCONFIG} -m on the directory where the
new library is installed (usually
PREFIX/lib) during
post-install target to register it into the
shared library cache. This variable, when defined, will also
facilitate addition of an appropriate
@exec /sbin/ldconfig -m and
@unexec /sbin/ldconfig -R pair into your
pkg-plist file, so that a user who installed
the package can start using the shared library immediately and
deinstallation will not cause the system to still believe the
library is there.If you need, you can override default location where the new
library is installed by defining LDCONFIG_DIRS
make variable, which should contain a list of directories into which
shared libraries are to be installed. For example if your port
installs shared libraries into
PREFIX/lib/foo and
PREFIX/lib/bar directories
you could use the following in your
Makefile:INSTALLS_SHLIB= yes
LDCONFIG_DIRS= %%PREFIX%%/lib/foo %%PREFIX%%/lib/barNote that content of LDCONFIG_DIRS is passed
through &man.sed.1; just like the rest of pkg-plist,
so PLIST_SUB substitutions also apply here. It is
recommended that you use %%PREFIX%% for
PREFIX, %%LOCALBASE%% for
LOCALBASE and %%X11BASE%% for
X11BASE.MASTERDIRIf your port needs to build slightly different versions of
packages by having a variable (for instance, resolution, or paper
size) take different values, create one subdirectory per package to
make it easier for users to see what to do, but try to share as many
files as possible between ports. Typically you only need a very short
Makefile in all but one of the directories if you
use variables cleverly. In the sole Makefiles,
you can use MASTERDIR to specify the directory
where the rest of the files are. Also, use a variable as part of
PKGNAMESUFFIX so
the packages will have different names.This will be best demonstrated by an example. This is part of
japanese/xdvi300/Makefile;PORTNAME= xdvi
PORTVERSION= 17
PKGNAMEPREFIX= ja-
PKGNAMESUFFIX= ${RESOLUTION}
:
# default
RESOLUTION?= 300
.if ${RESOLUTION} != 118 && ${RESOLUTION} != 240 && \
${RESOLUTION} != 300 && ${RESOLUTION} != 400
@${ECHO} "Error: invalid value for RESOLUTION: \"${RESOLUTION}\""
@${ECHO} "Possible values are: 118, 240, 300 (default) and 400."
@${FALSE}
.endifjapanese/xdvi300 also has all the regular
patches, package files, etc. If you type make
there, it will take the default value for the resolution (300) and
build the port normally.As for other resolutions, this is the entirexdvi118/Makefile:RESOLUTION= 118
MASTERDIR= ${.CURDIR}/../xdvi300
.include ${MASTERDIR}/Makefile(xdvi240/Makefile and
xdvi400/Makefile are similar). The
MASTERDIR definition tells
bsd.port.mk that the regular set of
subdirectories like FILESDIR and
SCRIPTDIR are to be found under
xdvi300. The RESOLUTION=118
line will override the RESOLUTION=300 line in
xdvi300/Makefile and the port will be built with
resolution set to 118.Shared library versionsPlease read our policy on
shared library versioning to understand what to do with
shared library versions in general. Do not blindly assume software
authors know what they are doing; many of them do not. It is very
important that these details are carefully considered, as we have
quite a unique situation where we are trying to have dozens of
potentially incompatible software pairs co-exist. Careless port
imports have caused great trouble regarding shared libraries in the
past (ever wondered why the port jpeg-6b has a
shared library version of 9?). If in doubt, send a message to the
&a.ports;. Most of the time, your job ends by determining the right
shared library version and making appropriate patches to implement
it.ManpagesThe MAN[1-9LN] variables will automatically add
any manpages to pkg-plist (this means you must
not list manpages in the
pkg-plist—see generating PLIST for more). It also
makes the install stage automatically compress or uncompress manpages
depending on the setting of NOMANCOMPRESS in
/etc/make.conf.If your port tries to install multiple names for manpages using
symlinks or hardlinks, you must use the MLINKS
variable to identify these. The link installed by your port will
be destroyed and recreated by bsd.port.mk
to make sure it points to the correct file. Any manpages
listed in MLINKS must not be listed in the
pkg-plist.To specify whether the manpages are compressed upon installation,
use the MANCOMPRESSED variable. This variable can
take three values, yes, no and
maybe. yes means manpages are
already installed compressed, no means they are
not, and maybe means the software already respects
the value of NOMANCOMPRESS so
bsd.port.mk does not have to do anything
special.MANCOMPRESSED is automatically set to
yes if USE_IMAKE is set and
NO_INSTALL_MANPAGES is not set, and to
no otherwise. You do not have to explicitly define
it unless the default is not suitable for your port.If your port anchors its man tree somewhere other than
PREFIX, you can use the
MANPREFIX to set it. Also, if only manpages in
certain sections go in a non-standard place, such as some Perl modules
ports, you can set individual man paths using
MANsectPREFIX (where
sect is one of 1-9,
L or N).If your manpages go to language-specific subdirectories, set the
name of the languages to MANLANG. The value of
this variable defaults to "" (i.e., English
only).Here is an example that puts it all together.MAN1= foo.1
MAN3= bar.3
MAN4= baz.4
MLINKS= foo.1 alt-name.8
MANLANG= "" ja
MAN3PREFIX= ${PREFIX}/share/foobar
MANCOMPRESSED= yesThis states that six files are installed by this port;${PREFIX}/man/man1/foo.1.gz
${PREFIX}/man/ja/man1/foo.1.gz
${PREFIX}/share/foobar/man/man3/bar.3.gz
${PREFIX}/share/foobar/man/ja/man3/bar.3.gz
${PREFIX}/man/man4/baz.4.gz
${PREFIX}/man/ja/man4/baz.4.gzAdditionally ${PREFIX}/man/man8/alt-name.8.gz
may or may not be installed by your port. Regardless, a
symlink will be made to join the foo(1) manpage and
alt-name(8) manpage.Ports that require MotifThere are many programs that require a Motif library (available
from several commercial vendors, while there is a free clone reported
to be able to run many applications in
x11-toolkits/lesstif) to compile. Since it is a
popular toolkit and their licenses usually permit redistribution of
statically linked binaries, we have made special provisions for
handling ports that require Motif in a way that we can easily compile
binaries linked either dynamically (for people who are compiling from
the port) or statically (for people who distribute packages).REQUIRES_MOTIFIf your port requires Motif, define this variable in the
Makefile. This will prevent people who do not own a copy of Motif
from even attempting to build it.MOTIFLIBThis variable will be set by bsd.port.mk to
be the appropriate reference to the Motif library. Please patch the
source to use this wherever the Motif library is referenced in the
Makefile or
Imakefile.There are two common cases:If the port refers to the Motif library as
-lXm in its Makefile or
Imakefile, simply substitute
${MOTIFLIB} for it.If the port uses XmClientLibs in its
Imakefile, change it to
${MOTIFLIB} ${XTOOLLIB}
${XLIB}.Note that MOTIFLIB (usually) expands to
-L/usr/X11R6/lib -lXm or
/usr/X11R6/lib/libXm.a, so there is no need to
add -L or -l in front.X11 fontsIf your port installs fonts for the X Window system, put them in
X11BASE/lib/X11/fonts/local.
This directory is new to XFree86 release 3.3.3. If it does not exist,
please create it, and print out a message urging the user to update
their XFree86 to 3.3.3 or newer, or at least add this directory to the
font path in /etc/XF86Config.Info filesThe new version of texinfo (included in 2.2.2-RELEASE and onwards)
contains a utility called install-info to add and
delete entries to the dir file. If your port
installs any info documents, please follow these instructions so your
port/package will correctly update the user's
PREFIX/info/dir file. (Sorry
for the length of this section, but is it imperative to weave all the
info files together. If done correctly, it will produce a
beautiful listing, so please bear with me!First, this is what you (as a porter) need to know&prompt.user; install-info --help
install-info [OPTION]... [INFO-FILE [DIR-FILE]]
Install INFO-FILE in the Info directory file DIR-FILE.
Options:
--delete Delete existing entries in INFO-FILE;
don't insert any new entries.
:
--entry=TEXT Insert TEXT as an Info directory entry.
:
--section=SEC Put this file's entries in section SEC of the directory. :This program will not actually install info
files; it merely inserts or deletes entries in the
dir file.Here's a seven-step procedure to convert ports to use
install-info.
editors/emacs will be used as an
example.Look at the texinfo sources and make a patch to insert
@dircategory and @direntry
statements to files that do not have them. This is part of my
patch:--- ./man/vip.texi.org Fri Jun 16 15:31:11 1995
+++ ./man/vip.texi Tue May 20 01:28:33 1997
@@ -2,6 +2,10 @@
@setfilename ../info/vip
@settitle VIP
+@dircategory The Emacs editor and associated tools
+@direntry
+* VIP: (vip). A VI-emulation for Emacs.
+@end direntry
@iftex
@finalout
:The format should be self-explanatory. Many authors leave a
dir file in the source tree that contains all
the entries you need, so look around before you try to write your
own. Also, make sure you look into related ports and make the
section names and entry indentations consistent (we recommend that
all entry text start at the 4th tab stop).Note that you can put only one info entry per file because
of a bug in install-info --delete that
deletes only the first entry if you specify multiple entries in
the @direntry section.You can give the dir entries to
install-info as arguments
( and ) instead
of patching the texinfo sources. This probably is not a good
idea for ports because you need to duplicate the same information
in three places
(Makefile and
@exec/@unexec of
pkg-plist; see below). However, if you have
Japanese (or other multibyte encoding) info files, you will have
to use the extra arguments to install-info
because makeinfo cannot handle those texinfo
sources. (See Makefile and
pkg-plist of japanese/skk
for examples on how to do this).Go back to the port directory and do a make clean;
make and verify that the info files are regenerated
from the texinfo sources. Since the texinfo sources are newer than
the info files, they should be rebuilt when you type
make; but many Makefiles
do not include correct dependencies for info files. In
emacs' case, it was necessary to patch the main
Makefile.in so it would descend into the
man subdirectory to rebuild the info
pages.--- ./Makefile.in.org Mon Aug 19 21:12:19 1996
+++ ./Makefile.in Tue Apr 15 00:15:28 1997
@@ -184,7 +184,7 @@
# Subdirectories to make recursively. `lisp' is not included
# because the compiled lisp files are part of the distribution
# and you cannot remake them without installing Emacs first.
-SUBDIR = lib-src src
+SUBDIR = lib-src src man
# The makefiles of the directories in $SUBDIR.
SUBDIR_MAKEFILES = lib-src/Makefile man/Makefile src/Makefile oldXMenu/Makefile
lwlib/Makefile
--- ./man/Makefile.in.org Thu Jun 27 15:27:19 1996
+++ ./man/Makefile.in Tue Apr 15 00:29:52 1997
@@ -66,6 +66,7 @@
${srcdir}/gnu1.texi \
${srcdir}/glossary.texi
+all: info
info: $(INFO_TARGETS)
dvi: $(DVI_TARGETS)The second hunk was necessary because the default target in
the man subdir is called
info, while the main
Makefile wants to call
all. The installation of the
info info file was also removed because we
already have one with the same name in
/usr/share/info (that patch is not shown
here).If there is a place in the Makefile that
is installing the dir file, delete it. Your
port may not be doing it. Also, remove any commands that are
otherwise mucking around with the dir
file.--- ./Makefile.in.org Mon Aug 19 21:12:19 1996
+++ ./Makefile.in Mon Apr 14 23:38:07 1997
@@ -368,14 +368,8 @@
if [ `(cd ${srcdir}/info && /bin/pwd)` != `(cd ${infodir} && /bin/pwd)` ]; \
then \
(cd ${infodir}; \
- if [ -f dir ]; then \
- if [ ! -f dir.old ]; then mv -f dir dir.old; \
- else mv -f dir dir.bak; fi; \
- fi; \
cd ${srcdir}/info ; \
- (cd $${thisdir}; ${INSTALL_DATA} ${srcdir}/info/dir ${infodir}/dir);
\
- (cd $${thisdir}; chmod a+r ${infodir}/dir); \
for f in ccmode* cl* dired-x* ediff* emacs* forms* gnus* info* message* mh-e* sc* vip*; do \
(cd $${thisdir}; \
${INSTALL_DATA} ${srcdir}/info/$$f ${infodir}/$$f; \
chmod a+r ${infodir}/$$f); \(This step is only necessary if you are modifying an existing
port.) Take a look at pkg-plist and delete
anything that is trying to patch up info/dir.
They may be in pkg-install or some other
file, so search extensively.Index: pkg-plist
===================================================================
RCS file: /usr/cvs/ports/editors/emacs/pkg-plist,v
retrieving revision 1.15
diff -u -r1.15 pkg-plist
--- pkg-plist 1997/03/04 08:04:00 1.15
+++ pkg-plist 1997/04/15 06:32:12
@@ -15,9 +15,6 @@
man/man1/emacs.1.gz
man/man1/etags.1.gz
man/man1/ctags.1.gz
-@unexec cp %D/info/dir %D/info/dir.bak
-info/dir
-@unexec cp %D/info/dir.bak %D/info/dir
info/cl
info/cl-1
info/cl-2Add a post-install target to the
Makefile to call
install-info with the installed
info files. (It is no longer necessary to create the
dir file yourself;
install-info automatically creates this
file if it does not exist.)Index: Makefile
===================================================================
RCS file: /usr/cvs/ports/editors/emacs/Makefile,v
retrieving revision 1.26
diff -u -r1.26 Makefile
--- Makefile 1996/11/19 13:14:40 1.26
+++ Makefile 1997/05/20 10:25:09 1.28
@@ -20,5 +20,8 @@
post-install:
.for file in emacs-19.34 emacsclient etags ctags b2m
strip ${PREFIX}/bin/${file}
.endfor
+.for info in emacs vip viper forms gnus mh-e cl sc dired-x ediff ccmode
+ install-info ${PREFIX}/info/${info} ${PREFIX}/info/dir
+.endfor
.include <bsd.port.mk>Edit pkg-plist and add equivalent
@exec statements and also
@unexec for
pkg_delete.Index: pkg-plist
===================================================================
RCS file: /usr/cvs/ports/editors/emacs/pkg-plist,v
retrieving revision 1.15
diff -u -r1.15 pkg-plist
--- pkg-plist 1997/03/04 08:04:00 1.15
+++ pkg-plist 1997/05/20 10:25:12 1.17
@@ -16,7 +14,14 @@
man/man1/etags.1.gz
man/man1/ctags.1.gz
+@unexec install-info --delete %D/info/emacs %D/info/dir
:
+@unexec install-info --delete %D/info/ccmode %D/info/dir
info/cl
info/cl-1
@@ -87,6 +94,18 @@
info/viper-3
info/viper-4
+@exec install-info %D/info/emacs %D/info/dir
:
+@exec install-info %D/info/ccmode %D/info/dir
libexec/emacs/19.34/i386--freebsd/cvtmail
libexec/emacs/19.34/i386--freebsd/digest-docThe @unexec install-info --delete
commands have to be listed before the info files themselves so
they can read the files. Also, the @exec
install-info commands have to be after the info
files and the @exec command that creates the
the dir file.Test and admire your
work. :-). Check the
dir file before and after each step.The pkg-* filesThere are some tricks we have not mentioned yet about the
pkg-* files
that come in handy sometimes.pkg-messageIf you need to display a message to the installer, you may place
the message in pkg-message. This capability is
often useful to display additional installation steps to be taken
after a pkg_add or to display licensing
information.The pkg-message file does not need to be
added to pkg-plist. Also, it will not get
automatically printed if the user is using the port, not the
package, so you should probably display it from the
post-install target yourself.pkg-installIf your port needs to execute commands when the binary package
is installed with pkg_add you can do this via the
pkg-install script. This script will
automatically be added to the package, and will be run twice by
pkg_add. The first time as
${SH} pkg-install ${PKGNAME}
PRE-INSTALL and the second time as
${SH} pkg-install ${PKGNAME} POST-INSTALL.
$2 can be tested to determine which mode
the script is being run in. The PKG_PREFIX
environmental variable will be set to the package installation
directory. See &man.pkg.add.1; for
additional information.This script is not run automatically if you install the port
with make install. If you are depending on it
being run, you will have to explicitly call it from your port's
Makefile.pkg-reqIf your port needs to determine if it should install or not, you
- can create a pkg-req “requirements”
+ can create a pkg-reqrequirements
script. It will be invoked automatically at
installation/deinstallation time to determine whether or not
installation/deinstallation should proceed.The script will be run at installation time by
pkg_add as
pkg-req ${PKGNAME} INSTALL.
At deinstallation time it will be run by
pkg_delete as
pkg-req ${PKGNAME} DEINSTALL.Changing pkg-plist based on make
variablesSome ports, particularly the p5- ports, need to change their
pkg-plist depending on what options they are
configured with (or version of perl, in the case of p5- ports). To
make this easy, any instances in the pkg-plist of
%%OSREL%%, %%PERL_VER%%, and
%%PERL_VERSION%% will be substituted for
appropriately. The value of %%OSREL%% is the
numeric revision of the operating system (e.g.,
2.2.7). %%PERL_VERSION%% is
the full version number of perl (e.g., 5.00502)
and %%PERL_VER%% is the perl version number minus
the patchlevel (e.g., 5.005).If you need to make other substitutions, you can set the
PLIST_SUB variable with a list of
VAR=VALUE
pairs and instances of
%%VAR%%' will be
substituted with VALUE in the
pkg-plist.For instance, if you have a port that installs many files in a
version-specific subdirectory, you can put something like
OCTAVE_VERSION= 2.0.13
PLIST_SUB= OCTAVE_VERSION=${OCTAVE_VERSION}
in the Makefile and use
%%OCTAVE_VERSION%% wherever the version shows up
in pkg-plist. That way, when you upgrade the port,
you will not have to change dozens (or in some cases, hundreds) of
lines in the pkg-plist.This substitution (as well as addition of any man pages) will be done between
the do-install and
post-install targets, by reading from
PLIST and writing to TMPPLIST
(default:
WRKDIR/.PLIST.mktmp). So if
your port builds PLIST on the fly, do so in or
before do-install. Also, if your port
needs to edit the resulting file, do so in
post-install to a file named
TMPPLIST.Changing the names of
pkg-* filesAll the names of pkg-* files
are defined using variables so you can change them in your
Makefile if need be. This is especially useful
when you are sharing the same pkg-* files
among several ports or have to write to one of the above files (see
writing to places other than
WRKDIR for why it is a bad idea to write
directly in to the pkg-* subdirectory).Here is a list of variable names and their default
values. (PKGDIR defaults to
${MASTERDIR}.)VariableDefault valueCOMMENT${PKGDIR}/pkg-commentDESCR${PKGDIR}/pkg-descrPLIST${PKGDIR}/pkg-plistPKGINSTALL${PKGDIR}/pkg-installPKGDEINSTALL${PKGDIR}/pkg-deinstallPKGREQ${PKGDIR}/pkg-reqPKGMESSAGE${PKGDIR}/pkg-messagePlease change these variables rather than overriding
PKG_ARGS. If you change
PKG_ARGS, those files will not correctly be
installed in /var/db/pkg upon install from a
port.Licensing ProblemsSome software packages have restrictive licenses or can be in
violation of the law in some countries (such as violating a patent).
What we can do with
them varies a lot, depending on the exact wordings of the respective
licenses.It is your responsibility as a porter to read the licensing
terms of the software and make sure that the FreeBSD project will
not be held accountable for violating them by redistributing the
source or compiled binaries either via FTP or CDROM. If in doubt,
please contact the &a.ports;.There are two variables you can set in the Makefile to handle the
situations that arise frequently:
- If the port has a “do not sell for profit” type of
+ If the port has a do not sell for profit type of
license, set the variable NO_CDROM to a string
describing the reason why. We will make sure such ports will not go
into the CDROM come release time. The distfile and package will
still be available via FTP.If the resulting package needs to be built uniquely for each
site, or the resulting binary package cannot be distributed due to
licensing; set the variable NO_PACKAGE to a
string describing the reason why. We will make sure such packages
will not go on the FTP site, nor into the CDROM come release time.
The distfile will still be included on both however.If the port has legal restrictions on who can use it (e.g.,
- patented stuff) or has a “no commercial use” license,
+ patented stuff) or has a no commercial use license,
set the variable RESTRICTED to be the string
describing the reason why. For such ports, the distfiles/packages
will not be available even from our FTP sites.The GNU General Public License (GPL), both version 1 and 2,
should not be a problem for ports.If you are a committer, make sure you update the
ports/LEGAL file too.UpgradingWhen you notice that a port is out of date compared to the latest
version from the original authors, first make sure you have the latest
port. You can find them in the
ports/ports-current directory of the FTP mirror
sites. You may also use CVSup to keep your whole ports collection
up-to-date, as described in the Handbook.The next step is to send a mail to the maintainer, if one is
listed in the port's Makefile. That person may
already be working on an upgrade, or have a reason to not upgrade the
port right now (because of, for example, stability problems of the new
version).If the maintainer asks you to do the upgrade or there is not any
such person to begin with, please make the upgrade and send the
recursive diff (either unified or context diff is fine, but port
committers appear to prefer unified diff more) of the new and old
ports directories to us (e.g., if your modified port directory is
called superedit and the original as in our tree
is superedit.bak, then send us the result of
diff -ruN superedit.bak superedit). Please examine
the output to make sure all the changes make sense. The best way to
send us the diff is by including it via &man.send-pr.1; (category
ports). Please mention any added or deleted files
in the message, as they have to be explicitly specified to CVS when
doing a commit. If the diff is more than about 20KB, please compress
and uuencode it; otherwise, just include it in the PR as is.Once again, please use &man.diff.1; and not &man.shar.1; to send
updates to existing ports!Dos and Don'tsHere is a list of common dos and don'ts that you encounter during
the porting process.You should check your own port against this list,
but you can also check ports in the PR database that others have
submitted. Submit any comments on ports you check as described in
Bug Reports and General
Commentary. Checking ports in the PR database will both make
it faster for us to commit them, and prove that you know what you are
doing.Strip BinariesDo strip binaries. If the original source already strips the
binaries, fine; otherwise you should add a
post-install rule to it yourself. Here is an
example:post-install:
strip ${PREFIX}/bin/xdlUse the &man.file.1; command on the installed executable to
check whether the binary is stripped or not. If it does not say
not stripped, it is stripped.INSTALL_* macrosDo use the macros provided in bsd.port.mk
to ensure correct modes and ownership of files in your own
*-install targets.INSTALL_PROGRAM is a command to install
binary executables.INSTALL_SCRIPT is a command to install
executable scripts.INSTALL_DATA is a command to install
sharable data.INSTALL_MAN is a command to install
manpages and other documentation (it does not compress
anything).These are basically the install command with
all the appropriate flags. See below for an example on how to use
them.WRKDIRDo not write anything to files outside
WRKDIR. WRKDIR is the only
place that is guaranteed to be writable during the port build (see
compiling ports from CDROM for an
example of building ports from a read-only tree). If you need to
modify one of the pkg-*
files, do so by redefining a variable, not by
writing over it.WRKDIRPREFIXMake sure your port honors WRKDIRPREFIX.
Most ports do not have to worry about this. In particular, if you
are referring to a WRKDIR of another port, note
that the correct location is
WRKDIRPREFIXPORTSDIR/subdir/name/work not PORTSDIR/subdir/name/work or .CURDIR/../../subdir/name/work or some such.Also, if you are defining WRKDIR yourself,
make sure you prepend
${WRKDIRPREFIX}${.CURDIR} in the
front.Differentiating operating systems and OS versionsYou may come across code that needs modifications or conditional
compilation based upon what version of Unix it is running under. If
you need to make such changes to the code for conditional
compilation, make sure you make the changes as general as possible
so that we can back-port code to FreeBSD 1.x systems and cross-port
to other BSD systems such as 4.4BSD from CSRG, BSD/386, 386BSD,
NetBSD, and OpenBSD.The preferred way to tell 4.3BSD/Reno (1990) and newer versions
of the BSD code apart is by using the BSD macro
defined in <sys/param.h>. Hopefully that
file is already included; if not, add the code:#if (defined(__unix__) || defined(unix)) && !defined(USG)
#include <sys/param.h>
#endifto the proper place in the .c file. We
believe that every system that defines these two symbols has
sys/param.h. If you find a system that
does not, we would like to know. Please send mail to the
&a.ports;.Another way is to use the GNU Autoconf style of doing
this:#ifdef HAVE_SYS_PARAM_H
#include <sys/param.h>
#endifDo not forget to add -DHAVE_SYS_PARAM_H to the
CFLAGS in the Makefile for
this method.Once you have sys/param.h included, you may
use:#if (defined(BSD) && (BSD >= 199103))to detect if the code is being compiled on a 4.3 Net2 code base
or newer (e.g. FreeBSD 1.x, 4.3/Reno, NetBSD 0.9, 386BSD, BSD/386
1.1 and below).Use:#if (defined(BSD) && (BSD >= 199306))to detect if the code is being compiled on a 4.4 code base or
newer (e.g. FreeBSD 2.x, 4.4, NetBSD 1.0, BSD/386 2.0 or
above).The value of the BSD macro is
199506 for the 4.4BSD-Lite2 code base. This is
stated for informational purposes only. It should not be used to
distinguish between versions of FreeBSD based only on 4.4-Lite vs.
versions that have merged in changes from 4.4-Lite2. The
__FreeBSD__ macro should be used instead.Use sparingly:__FreeBSD__ is defined in all versions of
FreeBSD. Use it if the change you are making
only affects FreeBSD. Porting gotchas like
the use of sys_errlist[] vs
strerror() are Berkeleyisms, not FreeBSD
changes.In FreeBSD 2.x, __FreeBSD__ is defined to
be 2. In earlier versions, it is
1. Later versions will bump it to match
their major version number.If you need to tell the difference between a FreeBSD 1.x
system and a FreeBSD 2.x or 3.x system, usually the right answer
is to use the BSD macros described above. If
there actually is a FreeBSD specific change (such as special
shared library options when using ld) then it
is OK to use __FreeBSD__ and #if
__FreeBSD__ > 1 to detect a FreeBSD 2.x and later
system. If you need more granularity in detecting FreeBSD
systems since 2.0-RELEASE you can use the following:#if __FreeBSD__ >= 2
#include <osreldate.h>
# if __FreeBSD_version >= 199504
/* 2.0.5+ release specific code here */
# endif
#endifRelease__FreeBSD_version2.0-RELEASE1194112.1-CURRENT199501, 1995032.0.5-RELEASE1995042.2-CURRENT before 2.11995082.1.0-RELEASE1995112.2-CURRENT before 2.1.51995122.1.5-RELEASE1996072.2-CURRENT before 2.1.61996082.1.6-RELEASE1996122.1.7-RELEASE1996122.2-RELEASE2200002.2.1-RELEASE220000 (no change)2.2-STABLE after 2.2.1-RELEASE220000 (no change)2.2-STABLE after texinfo-3.92210012.2-STABLE after top2210022.2.2-RELEASE2220002.2-STABLE after 2.2.2-RELEASE2220012.2.5-RELEASE2250002.2-STABLE after 2.2.5-RELEASE2250012.2-STABLE after ldconfig -R merge2250022.2.6-RELEASE2260002.2.7-RELEASE2270002.2-STABLE after 2.2.7-RELEASE2270012.2-STABLE after &man.semctl.2; change2270022.2.8-RELEASE2280002.2-STABLE after 2.2.8-RELEASE2280013.0-CURRENT before &man.mount.2; change3000003.0-CURRENT after &man.mount.2; change3000013.0-CURRENT after &man.semctl.2; change3000023.0-CURRENT after ioctl arg changes3000033.0-CURRENT after ELF conversion3000043.0-RELEASE3000053.0-CURRENT after 3.0-RELEASE3000063.0-STABLE after 3/4 branch3000073.1-RELEASE3100003.1-STABLE after 3.1-RELEASE3100013.1-STABLE after C++ constructor/destructor order
change3100023.2-RELEASE3200003.2-STABLE3200013.2-STABLE after binary-incompatible IPFW and
socket changes3200023.3-RELEASE3300003.3-STABLE3300013.3-STABLE after adding &man.mkstemp.3;
to libc3300023.4-RELEASE3400003.4-STABLE3400014.0-CURRENT after 3.4 branch4000004.0-CURRENT after change in dynamic linker
handling4000014.0-CURRENT after C++ constructor/destructor
order change4000024.0-CURRENT after functioning &man.dladdr.3;4000034.0-CURRENT after __deregister_frame_info dynamic
linker bug fix (also 4.0-CURRENT after EGCS 1.1.2
integration)
4000044.0-CURRENT after &man.suser.9; API change
(also 4.0-CURRENT after newbus)4000054.0-CURRENT after cdevsw registration change4000064.0-CURRENT after the addition of so_cred for
socket level credentials4000074.0-CURRENT after the addition of a poll syscall
wrapper to libc_r4000084.0-CURRENT after the change of the kernel's
dev_t type to struct
specinfo pointer4000094.0-CURRENT after fixing a hole
in &man.jail.2;4000104.0-CURRENT after the sigset_t
datatype change4000114.0-CURRENT after the cutover to the GCC 2.95.2
compiler4000124.0-CURRENT after adding pluggable linux-mode
ioctl handlers4000134.0-CURRENT after importing OpenSSL4000144.0-CURRENT after the C++ ABI change in GCC 2.95.2
from -fvtable-thunks to -fno-vtable-thunks by
default4000154.0-CURRENT after importing OpenSSH4000164.0-RELEASE4000174.0-STABLE after 4.0-RELEASE4000184.0-STABLE after merging libxpg4 code into
libc.4000204.0-STABLE after upgrading Binutils to 2.10.0, ELF
branding changes, and tcsh in the base system.4000214.1-RELEASE4100004.1-STABLE after 4.1-RELEASE4100014.1-STABLE after &man.setproctitle.3; moved from
libutil to libc.4100024.1.1-RELEASE4110004.1.1-STABLE after 4.1.1-RELEASE4110014.2-RELEASE4200004.2-STABLE after combining libgcc.a and
libgcc_r.a, and associated GCC linkage changes.4200014.3-RELEASE4300004.3-STABLE after wint_t introduction.4300014.3-STABLE after PCI powerstate API merge.4300024.4-RELEASE4400005.0-CURRENT5000005.0-CURRENT after adding addition ELF header fields,
and changing our ELF binary branding method.5000015.0-CURRENT after kld metadata changes.5000025.0-CURRENT after buf/bio changes.5000035.0-CURRENT after binutils upgrade.5000045.0-CURRENT after merging libxpg4 code into
libc and after TASKQ interface introduction.5000055.0-CURRENT after the addition of AGP
interfaces.5000065.0-CURRENT after Perl upgrade to 5.6.05000075.0-CURRENT after the update of KAME code to
2000/07 sources.5000085.0-CURRENT after ether_ifattach() and
ether_ifdetach() changes.5000095.0-CURRENT after changing mtree defaults
back to original variant, adding -L to follow
symlinks.5000105.0-CURRENT after kqueue API changed.5000115.0-CURRENT after &man.setproctitle.3; moved from
libutil to libc.5000125.0-CURRENT after the first SMPng commit.5000135.0-CURRENT after <sys/select.h> moved to
<sys/selinfo.h>.5000145.0-CURRENT after combining libgcc.a and
libgcc_r.a, and associated GCC linkage changes.5000155.0-CURRENT after change allowing libc and libc_r
to be linked together, deprecating -pthread option.5000165.0-CURRENT after switch from struct ucred to
struct xucred to stabilize kernel-exported API for
mountd et al.5000175.0-CURRENT after addition of CPUTYPE make variable
for controlling CPU-specific optimizations.5000185.0-CURRENT after moving machine/ioctl_fd.h to
sys/fdcio.h5000195.0-CURRENT after locale names renaming.5000205.0-CURRENT after Bzip2 import.5000215.0-CURRENT after SSE support.5000225.0-CURRENT after KSE Milestone 2.5000235.0-CURRENT after d_thread_t.5000245.0-CURRENT after ABI change for discriptor
and creds passing on 64 bit platforms.5000255.0-CURRENT after moving to XFree86 4 by default for
package builds, and after the new libc strnstr() function
was added.5000265.0-CURRENT after the new libc strcasestr() function
was added.500027Note that 2.2-STABLE sometimes identifies itself as
- “2.2.5-STABLE” after the 2.2.5-RELEASE. The pattern
+ 2.2.5-STABLE after the 2.2.5-RELEASE. The pattern
used to be year followed by the month, but we decided to change it
to a more straightforward major/minor system starting from 2.2.
This is because the parallel development on several branches made
it infeasible to classify the releases simply by their real
release dates. If you are making a port now, you do not have to
worry about old -CURRENTs; they are listed here just for your
reference.In the hundreds of ports that have been done, there have only
been one or two cases where __FreeBSD__ should
have been used. Just because an earlier port screwed up and used it
in the wrong place does not mean you should do so too.Writing something after
bsd.port.mkDo not write anything after the .include
<bsd.port.mk> line. It usually can be avoided by
including bsd.port.pre.mk somewhere in the
middle of your Makefile and
bsd.port.post.mk at the end.You need to include either the
pre.mk/post.mk pair or
bsd.port.mk only; do not mix these two.bsd.port.pre.mk only defines a few
variables, which can be used in tests in the
Makefile, bsd.port.post.mk
defines the rest.Here are some important variables defined in
bsd.port.pre.mk (this is not the complete list,
please read bsd.port.mk for the complete
list).VariableDescriptionARCHThe architecture as returned by uname
-m (e.g., i386)OPSYSThe operating system type, as returned by
uname -s (e.g.,
FreeBSD)OSRELThe release version of the operating system (e.g.,
2.1.5 or
2.2.7)OSVERSIONThe numeric version of the operating system, same as
__FreeBSD_version.PORTOBJFORMATThe object format of the system
(aout or elf)LOCALBASE
- The base of the “local” tree (e.g.,
+ The base of the local tree (e.g.,
/usr/local/)X11BASE
- The base of the “X11” tree (e.g.,
+ The base of the X11 tree (e.g.,
/usr/X11R6)PREFIXWhere the port installs itself (see more on
PREFIX).If you have to define the variables
USE_IMAKE, USE_X_PREFIX, or
MASTERDIR, do so before including
bsd.port.pre.mk.Here are some examples of things you can write after
bsd.port.pre.mk:# no need to compile lang/perl5 if perl5 is already in system
.if ${OSVERSION} > 300003
BROKEN= perl is in system
.endif
# only one shlib version number for ELF
.if ${PORTOBJFORMAT} == "elf"
TCL_LIB_FILE= ${TCL_LIB}.${SHLIB_MAJOR}
.else
TCL_LIB_FILE= ${TCL_LIB}.${SHLIB_MAJOR}.${SHLIB_MINOR}
.endif
# software already makes link for ELF, but not for a.out
post-install:
.if ${PORTOBJFORMAT} == "aout"
${LN} -sf liblinpack.so.1.0 ${PREFIX}/lib/liblinpack.so
.endifInstall additional documentationIf your software has some documentation other than the standard
man and info pages that you think is useful for the user, install it
under PREFIX/share/doc.
This can be done, like the previous item, in the
post-install target.Create a new directory for your port. The directory name should
reflect what the port is. This usually means
PORTNAME. However, if you
think the user might want different versions of the port to be
installed at the same time, you can use the whole
PKGNAME.Make the installation dependent to the variable
NOPORTDOCS so that users can disable it in
/etc/make.conf, like this:post-install:
.if !defined(NOPORTDOCS)
${MKDIR} ${PREFIX}/share/doc/xv
${INSTALL_MAN} ${WRKSRC}/docs/xvdocs.ps ${PREFIX}/share/doc/xv
.endifAll documentation files and directories installed should
be included in pkg-plist with the
%%PORTDOCS%% prefix, for example:%%PORTDOCS%%share/doc/pure-ftpd/AUTHORS
%%PORTDOCS%%share/doc/pure-ftpd/CONTACT
%%PORTDOCS%%@dirrm share/doc/pure-ftpdYou can also use the pkg-message file to
display messages upon installation. See the using
pkg-message section for
details.pkg-message does not need to be added to
pkg-plist.DIST_SUBDIRDo not let your port clutter
/usr/ports/distfiles. If your port requires a
lot of files to be fetched, or contains a file that has a name that
might conflict with other ports (e.g.,
Makefile), set DIST_SUBDIR
to the name of the port (${PORTNAME} or
${PKGNAMEPREFIX}${PORTNAME}
should work fine). This will change
DISTDIR from the default
/usr/ports/distfiles to
/usr/ports/distfiles/DIST_SUBDIR,
and in effect puts everything that is required for your port into
that subdirectory.It will also look at the subdirectory with the same name on the
backup master site at ftp.FreeBSD.org.
(Setting DISTDIR explicitly in your
Makefile will not accomplish this, so please use
DIST_SUBDIR.)This does not affect the MASTER_SITES you
define in your Makefile.Package informationDo include package information, i.e.
pkg-comment, pkg-descr, and
pkg-plist.Note that these files are not used only for packaging anymore,
and are mandatory now, even if
NO_PACKAGE is set.RCS stringsDo not put RCS strings in patches. CVS will mangle them when we
put the files into the ports tree, and when we check them out again,
they will come out different and the patch will fail. RCS strings
are surrounded by dollar ($) signs, and
typically start with $Id or
$RCS.Recursive diffUsing the recurse () option to
diff to generate patches is fine, but please take
a look at the resulting patches to make sure you do not have any
unnecessary junk in there. In particular, diffs between two backup
files, Makefiles when the port uses
Imake or GNU configure, etc.,
are unnecessary and should be deleted. If you had to edit
configure.in and run
autoconf to regenerate
configure, do not take the diffs of
configure (it often grows to a few thousand
lines!); define USE_AUTOCONF=yes and take the
diffs of configure.in.Also, if you had to delete a file, then you can do it in the
post-extract target rather than as part of
the patch. Once you are happy with the resulting diff, please split
it up into one source file per patch file.PREFIXDo try to make your port install relative to
PREFIX. (The value of this variable will be set
to LOCALBASE (default
/usr/local), unless
USE_X_PREFIX or USE_IMAKE is
set, in which case it will be X11BASE (default
/usr/X11R6).)Not hard-coding /usr/local or
/usr/X11R6 anywhere in the source will make the
port much more flexible and able to cater to the needs of other
sites. For X ports that use imake, this is
automatic; otherwise, this can often be done by simply replacing the
occurrences of /usr/local (or
/usr/X11R6 for X ports that do not use imake)
in the various scripts/Makefiles in the port to read
PREFIX, as this variable is automatically passed
down to every stage of the build and install processes.Make sure your application isn't installing things in
/usr/local instead of PREFIX.
A quick test for this is to do this is:&prompt.root; make clean; make package PREFIX=/var/tmp/port-nameIf anything is installed outside of PREFIX,
making the package creation process will complain that it
can't find the files.This does not test for the existence of internal references,
or correct use of LOCALBASE for references to
files from other ports. Testing the installation in
/var/tmp/port-name
to do that that while you have it installed would do that.Do not set USE_X_PREFIX unless your port
truly requires it (i.e., it links against X libs or it needs to
reference files in X11BASE).The variable PREFIX can be reassigned in your
Makefile or in the user's environment.
However, it is strongly discouraged for individual ports to set this
variable explicitly in the Makefiles.Also, refer to programs/files from other ports with the
variables mentioned above, not explicit pathnames. For instance, if
your port requires a macro PAGER to be the full
pathname of less, use the compiler flag:
-DPAGER=\"${PREFIX}/bin/less\"
or
-DPAGER=\"${LOCALBASE}/bin/less\"
if this is an X port, instead of
-DPAGER=\"/usr/local/bin/less\". This way it will
have a better chance of working if the system administrator has
moved the whole `/usr/local' tree somewhere else.SubdirectoriesTry to let the port put things in the right subdirectories of
PREFIX. Some ports lump everything and put it in
the subdirectory with the port's name, which is incorrect. Also,
many ports put everything except binaries, header files and manual
pages in the a subdirectory of lib, which does
not bode well with the BSD paradigm. Many of the files should be
moved to one of the following: etc
(setup/configuration files), libexec
(executables started internally), sbin
(executables for superusers/managers), info
(documentation for info browser) or share
(architecture independent files). See man &man.hier.7; for details,
the rules governing
/usr pretty much apply to
/usr/local too. The exception are ports
- dealing with USENET “news”. They may use
+ dealing with USENET news. They may use
PREFIX/news as a destination
for their files.Cleaning up empty directoriesDo make your ports clean up after themselves when they are
deinstalled. This is usually accomplished by adding
@dirrm lines for all directories that are
specifically created by the port. You need to delete subdirectories
before you can delete parent directories. :
lib/X11/oneko/pixmaps/cat.xpm
lib/X11/oneko/sounds/cat.au
:
@dirrm lib/X11/oneko/pixmaps
@dirrm lib/X11/oneko/sounds
@dirrm lib/X11/onekoHowever, sometimes @dirrm will give you
errors because other ports also share the same subdirectory. You
can call rmdir from @unexec to
remove only empty directories without warning.@unexec rmdir %D/share/doc/gimp 2>/dev/null || trueThis will neither print any error messages nor cause
pkg_delete to exit abnormally even if
PREFIX/share/doc/gimp is not
empty due to other ports installing some files in there.UIDsIf your port requires a certain user to be on the installed
system, let the pkg-install script call
pw to create it automatically. Look at
net/cvsup-mirror for an example.If your port must use the same user/group ID number when it is
installed as a binary package as when it was compiled, then you must
choose a free UID from 50 to 99 and register it below. Look at
japanese/Wnn for an example.Make sure you do not use a UID already used by the system or
other ports. This is the current list of UIDs between 50 and
99.majordom:*:54:54:Majordomo Pseudo User:/usr/local/majordomo:/nonexistent
cyrus:*:60:60:the cyrus mail server:/nonexistent:/nonexistent
gnats:*:61:1:GNATS database owner:/usr/local/share/gnats/gnats-db:/bin/sh
uucp:*:66:66:UUCP pseudo-user:/var/spool/uucppublic:/usr/libexec/uucp/uucico
xten:*:67:67:X-10 daemon:/usr/local/xten:/nonexistent
pop:*:68:6:Post Office Owner (popper):/nonexistent:/nonexistent
wnn:*:69:7:Wnn:/nonexistent:/nonexistent
ifmail:*:70:66:Ifmail user:/nonexistent:/nonexistent
pgsql:*:70:70:PostgreSQL pseudo-user:/usr/local/pgsql:/bin/sh
ircd:*:72:72:IRCd hybrid:/nonexistent:/nonexistent
alias:*:81:81:QMail user:/var/qmail/alias:/nonexistent
qmaill:*:83:81:QMail user:/var/qmail:/nonexistent
qmaild:*:82:81:QMail user:/var/qmail:/nonexistent
qmailq:*:85:82:QMail user:/var/qmail:/nonexistent
qmails:*:87:82:QMail user:/var/qmail:/nonexistent
qmailp:*:84:81:QMail user:/var/qmail:/nonexistent
qmailr:*:86:82:QMail user:/var/qmail:/nonexistent
msql:*:87:87:mSQL-2 pseudo-user:/var/db/msqldb:/bin/sh
mysql:*:88:88:MySQL Daemon:/var/db/mysql:/sbin/nologin
vpopmail:*:89:89::0:0:User &:/usr/local/vpopmail:/nonexistent
smmsp:*:90:90:Sendmail Queue:/nonexistent:/nonexistentPlease include a notice when you submit a port (or an upgrade)
that reserves a new UID or GID in this range. This allows us to
keep the list of reserved IDs up to date.Do things rationallyThe Makefile should do things simply and
reasonably. If you can make it a couple of lines shorter or more
readable, then do so. Examples include using a make
.if construct instead of a shell
if construct, not redefining
do-extract if you can redefine
EXTRACT* instead, and using
GNU_CONFIGURE instead of CONFIGURE_ARGS
+= --prefix=${PREFIX}.Respect CFLAGSThe port should respect the CFLAGS variable.
If it does not, please add NO_PACKAGE=ignores
cflags to the Makefile.An example of a Makefile respecting
the CFLAGS variable follows. Note the
+=:CFLAGS += -Wall -WerrorHere is an example which does not respect the
CFLAGS variable:CFLAGS = -Wall -WerrorThe CFLAGS variable is defined on
FreeBSD systems in /etc/make.conf. The
first example appends additional flags to the
CFLAGS variable, preserving any system-wide
definitions. The second example clobbers anything previously
defined.Configuration filesIf your port requires some configuration files in
PREFIX/etc, do
not just install them and list them in
pkg-plist. That will cause
pkg_delete to delete files carefully edited by
the user and a new installation to wipe them out.Instead, install sample files with a suffix
(filename.sample
will work well) and print out a message pointing out that the
user has to copy and edit the file before the software can be made
to work.PortlintDo check your work with portlint
before you submit or commit it.FeedbackDo send applicable changes/patches to the original
author/maintainer for inclusion in next release of the code. This
will only make your job that much easier for the next
release.README.htmlDo not include the README.html file. This
file is not part of the cvs collection but is generated using the
make readme command.
MiscellaneaThe files pkg-comment,
pkg-descr, and pkg-plist
should each be double-checked. If you are reviewing a port and feel
they can be worded better, do so.Do not copy more copies of the GNU General Public License into
our system, please.Please be careful to note any legal issues! Do not let us
illegally distribute software!If you are stuck…Do look at existing examples and the
bsd.port.mk file before asking us questions!
;-)Do ask us questions if you have any trouble! Do not just beat
your head against a wall! :-)A Sample MakefileHere is a sample Makefile that you can use to
create a new port. Make sure you remove all the extra comments (ones
between brackets)!It is recommended that you follow this format (ordering of
variables, empty lines between sections, etc.). This format is
designed so that the most important information is easy to locate. We
recommend that you use portlint to check the
Makefile.[the header...just to make it easier for us to identify the ports.]
# New ports collection makefile for: xdvi
[the "version required" line is only needed when the PORTVERSION
variable is not specific enough to describe the port.]
# Date created: 26 May 1995
[this is the person who did the original port to FreeBSD, in particular, the
person who wrote the first version of this Makefile. Remember, this should
not be changed when upgrading the port later.]
# Whom: Satoshi Asami <asami@FreeBSD.org>
#
# $FreeBSD$
[ ^^^^^^^^^ This will be automatically replaced with RCS ID string by CVS
when it is committed to our repository. If upgrading a port, do not alter
this line back to "$FreeBSD$". CVS deals with it automatically.]
#
[section to describe the port itself and the master site - PORTNAME
and PORTVERSION are always first, followed by CATEGORIES,
and then MASTER_SITES, which can be followed by MASTER_SITE_SUBDIR.
PKGNAMEPREFIX and PKGNAMESUFFIX, if needed, will be after that.
Then comes DISTNAME, EXTRACT_SUFX and/or DISTFILES, and then
EXTRACT_ONLY, as necessary.]
PORTNAME= xdvi
PORTVERSION= 18.2
CATEGORIES= print
[do not forget the trailing slash ("/")!
if you are not using MASTER_SITE_* macros]
MASTER_SITES= ${MASTER_SITE_XCONTRIB}
MASTER_SITE_SUBDIR= applications
PKGNAMEPREFIX= ja-
DISTNAME= xdvi-pl18
[set this if the source is not in the standard ".tar.gz" form]
EXTRACT_SUFX= .tar.Z
[section for distributed patches -- can be empty]
PATCH_SITES= ftp://ftp.sra.co.jp/pub/X11/japanese/
PATCHFILES= xdvi-18.patch1.gz xdvi-18.patch2.gz
[maintainer; *mandatory*! This is the person (preferably with commit
privileges) whom a user can contact for questions and bug reports - this
person should be the porter or someone who can forward questions to the
original porter reasonably promptly. If you really do not want to have
your address here, set it to "ports@FreeBSD.org".]
MAINTAINER= asami@FreeBSD.org
[dependencies -- can be empty]
RUN_DEPENDS= gs:${PORTSDIR}/print/ghostscript
LIB_DEPENDS= Xpm.5:${PORTSDIR}/graphics/xpm
[this section is for other standard bsd.port.mk variables that do not
belong to any of the above]
[If it asks questions during configure, build, install...]
IS_INTERACTIVE= yes
[If it extracts to a directory other than ${DISTNAME}...]
WRKSRC= ${WRKDIR}/xdvi-new
[If the distributed patches were not made relative to ${WRKSRC}, you
may need to tweak this]
PATCH_DIST_STRIP= -p1
[If it requires a "configure" script generated by GNU autoconf to be run]
GNU_CONFIGURE= yes
[If it requires GNU make, not /usr/bin/make, to build...]
USE_GMAKE= yes
[If it is an X application and requires "xmkmf -a" to be run...]
USE_IMAKE= yes
[et cetera.]
[non-standard variables to be used in the rules below]
MY_FAVORITE_RESPONSE= "yeah, right"
[then the special rules, in the order they are called]
pre-fetch:
i go fetch something, yeah
post-patch:
i need to do something after patch, great
pre-install:
and then some more stuff before installing, wow
[and then the epilogue]
.include <bsd.port.mk>Automated package list creationFirst, make sure your port is almost complete, with only
pkg-plist missing. Create an empty
pkg-plist.&prompt.root; touch pkg-plistNext, create a new set of directories which your port can be
installed, and install any dependencies.&prompt.root; mtree -U -f /etc/mtree/BSD.local.dist -d -e -p /var/tmp/port-name
&prompt.root; make depends PREFIX=/var/tmp/port-nameStore the directory structure in a new file.&prompt.root; (cd /var/tmp/port-name && find * -type d) > OLD-DIRSIf your port honors PREFIX (which it should)
you can then install the port and create the package list.&prompt.root; make install PREFIX=/var/tmp/port-name
&prompt.root; (cd /var/tmp/port-name && find * \! -type d) > pkg-plistYou must also add any newly created directories to the packing
list.&prompt.root; (cd /var/tmp/port-name && find * -type d) | comm -13 OLD-DIRS - | sed -e 's#^#@dirrm #' >> pkg-plistFinally, you need to tidy up the packing list by hand; it isn't
all automated. Manual pages should be listed in
the port's Makefile under
MANn, and not in the
package list. User configuration files should be removed, or
installed as
filename.sample.
The info/dir file should not be listed
and appropriate install-info lines should
be added as noted in the info
files section. Any
libraries installed by the port should be listed as specified in the
shared libraries section.Package NamesThe following are the conventions you should follow in naming your
packages. This is to have our package directory easy to scan, as
there are already lots and lots of packages and users are going to
turn away if they hurt their eyes!The package name should look like
language_region-name-compiled.specifics-version.numbers.The package name is defined as
${PKGNAMEPREFIX}${PORTNAME}${PKGNAMESUFFIX}-${PORTVERSION}.
Make sure to set the variables to conform to that format.FreeBSD strives to support the native language of its users.
The language- part should be a two
letter abbreviation of the natural language defined by ISO-639 if
the port is specific to a certain language. Examples are
ja for Japanese, ru for
Russian, vi for Vietnamese,
zh for Chinese, ko for
Korean and de for German.If the port is specific to a certain region within the
language area, add the two letter country code as well.
Examples are en_US for US English and
fr_CH for Swiss French.The language- part should
be set in the PKGNAMEPREFIX variable.The first letter of name part
should be lowercase. (The rest of the name can contain
capital letters, so use your own discretion when you are
converting a software name that has some capital letters in it.)
There is a tradition of naming Perl 5 modules by
prepending p5- and converting the double-colon
separator to a hyphen; for example, the
Data::Dumper module becomes
p5-Data-Dumper. If the software in question
has numbers, hyphens, or underscores in its name, you may include
them as well (like kinput2).If the port can be built with different hardcoded defaults (usually
part of the directory name in a family of ports), the
-compiled.specifics part should state
the compiled-in defaults (the hyphen is optional). Examples are
papersize and font units.The compiled.specifics part
should be set in the PKGNAMESUFFIX
variable.The version string should follow a dash
(-) and be a period-separated list of
integers and single lowercase alphabetics. In particular,
it is not permissible to have another dash inside the
version string. The only exception is the string
pl (meaning `patchlevel'), which can be
used only when there are no major and
minor version numbers in the software. If the software
version has strings like "alpha", "beta", "rc", or "pre", take
the first letter and put it immediately after a period.
If the version string continues after those names, the
numbers should follow the single alphabet without an extra
period between them.The idea is to make it easier to sort ports by looking
at the version string. In particular, make sure version
number components are always delimited by a period, and
if the date is part of the string, use the
yyyy.mm.dd
format, not
dd.mm.yyyy
or the non-Y2K compliant
yy.mm.dd
format.Here are some (real) examples on how to convert the name
as called by the software authors to a suitable package
name:Distribution NamePKGNAMEPREFIXPORTNAMEPKGNAMESUFFIXPORTVERSIONReasonmule-2.2.2(empty)mule(empty)2.2.2No changes requiredXFree86-3.3.6(empty)XFree86(empty)3.3.6No changes requiredEmiClock-1.0.2(empty)emiclock(empty)1.0.2No uppercase names for single programsrdist-1.3alpha(empty)rdist(empty)1.3.aNo strings like alpha
allowedes-0.9-beta1(empty)es(empty)0.9.b1No strings like beta
allowedmailman-2.0rc3(empty)mailman(empty)2.0.r3No strings like rc
allowedv3.3beta021.src(empty)tiff(empty)3.3What the heck was that anyway?tvtwm(empty)tvtwm(empty)pl11Version string always requiredpiewm(empty)piewm(empty)1.0Version string always requiredxvgr-2.10pl1(empty)xvgr(empty)2.10.1pl allowed only when no
major/minor version numbersgawk-2.15.6ja-gawk(empty)2.15.6Japanese language versionpsutils-1.13(empty)psutils-letter1.13Papersize hardcoded at package build timepkfonts(empty)pkfonts3001.0Package for 300dpi fontsIf there is absolutely no trace of version information in the
original source and it is unlikely that the original author will ever
release another version, just set the version string to
1.0 (like the piewm example above). Otherwise, ask
the original author or use the date string
(yyyy.mm.dd)
as the version.CategoriesAs you already know, ports are classified in several categories.
But for this to work, it is important that porters and users understand
what each category is for and how we decide what to put in each
category.Current list of categoriesFirst, this is the current list of port categories. Those
marked with an asterisk (*) are
virtual categories—those that do not have
a corresponding subdirectory in the ports tree.For non-virtual categories, you will find a one-line
description in the pkg/COMMENT file in that
subdirectory (e.g.,
archivers/pkg/COMMENT).CategoryDescriptionafterstep*Ports to support the AfterStep window manager.archiversArchiving tools.astroAstronomical ports.audioSound support.benchmarksBenchmarking utilities.biologyBiology-related software.cadComputer aided design tools.chineseChinese language support.commsCommunication software. Mostly software to talk to
your serial port.convertersCharacter code converters.databasesDatabases.deskutilsThings that used to be on the desktop before
computers were invented.develDevelopment utilities. Do not put libraries here just
because they are libraries—unless they truly do not
belong anywhere else, they should not be in this
category.editorsGeneral editors. Specialized editors go in the section
for those tools (e.g., a mathematical-formula editor will go
in math).elisp*Emacs-lisp ports.emulatorsEmulators for other operating systems. Terminal
emulators do not belong
here—X-based ones should go to
x11 and text-based ones to either
comms or misc,
depending on the exact functionality.frenchFrench language support.ftpFTP client and server utilities. If your
port speaks both FTP and HTTP, put it in
ftp with a secondary
category of www.gamesGames.germanGerman language support.gnome*Ports from the GNU Object Model Environment (GNOME)
Project.graphicsGraphics utilities.hebrewHebrew language support.ircInternet Relay Chat utilities.ipv6*IPv6 related software.japaneseJapanese language support.javaJava language support.kde*Ports from the K Desktop Environment (KDE)
Project.koreanKorean language support.langProgramming languages.linux*Linux applications and support utilities.mailMail software.mathNumerical computation software and other utilities
for mathematics.mboneMBone applications.miscMiscellaneous utilities—basically things that
do not belong anywhere else. This is the only category
that should not appear with any other non-virtual category.
If you have misc with something else in
your CATEGORIES line, that means you can
safely delete misc and just put the port
in that other subdirectory!netMiscellaneous networking software.newsUSENET news software.offix*Ports from the OffiX suite.palmSoftware support for the 3Com Palm(tm) series.perl5*Ports that require perl version 5 to run.picobsdPorts to support PicoBSD.plan9*Various programs from Plan9.printPrinting software. Desktop publishing tools
(previewers, etc.) belong here too.python*Software written in python.ruby*Software written in ruby.russianRussian language support.scienceScientific ports that don't fit into other
categories such as astro,
biology and
math.securitySecurity utilities.shellsCommand line shells.sysutilsSystem utilities.tcl76*Ports that use Tcl version 7.6 to run.tcl80*Ports that use Tcl version 8.0 to run.tcl81*Ports that use Tcl version 8.1 to run.tcl82*Ports that use Tcl version 8.2 to run.textprocText processing utilities. It does not include
desktop publishing tools, which go to print/.tk42*Ports that use Tk version 4.2 to run.tk80*Ports that use Tk version 8.0 to run.tk81*Ports that use Tk version 8.1 to run.tk82*Ports that use Tk version 8.2 to run.tkstep80*Ports that use TkSTEP version 8.0 to run.ukrainianUkrainian language support.vietnameseVietnamese language support.windowmaker*Ports to support the WindowMaker window
managerwwwSoftware related to the World Wide Web. HTML language
support belongs here too.x11The X window system and friends. This category is only
for software that directly supports the window system. Do not
put regular X applications here. If your port is an X
application, define USE_XLIB (implied by
USE_IMAKE) and put it in the appropriate
categories. Also, many of them go into other
x11-* categories (see below).x11-clocksX11 clocks.x11-fmX11 file managers.x11-fontsX11 fonts and font utilities.x11-serversX11 servers.x11-toolkitsX11 toolkits.x11-wmX11 window managers.zope*Zope support.Choosing the right categoryAs many of the categories overlap, you often have to choose
which of the categories should be the primary category of your port.
There are several rules that govern this issue. Here is the list of
priorities, in decreasing order of precedence.Language specific categories always come first. For
example, if your port installs Japanese X11 fonts, then your
CATEGORIES line would read japanese
x11-fonts.Specific categories win over less-specific ones. For
instance, an HTML editor should be listed as www
editors, not the other way around. Also, you do not
need to list net when the port belongs to
any of irc, mail,
mbone, news,
security, or www.x11 is used as a secondary category only
when the primary category is a natural language. In particular,
you should not put x11 in the category line
for X applications.Emacs modes should be
placed in the same ports category as the application
supported by the mode, not in
editors. For example, an
Emacs mode to edit source
files of some programming language should go into
lang.
If your port truly does not belong anywhere else, put it in
misc.If you are not sure about the category, please put a comment to
that effect in your send-pr submission so we can
discuss it before we import it. If you are a committer, send a note
to the &a.ports; so we can discuss it first—too often new ports are
imported to the wrong category only to be moved right away.Changes to this document and the ports systemIf you maintain a lot of ports, you should consider following the
&a.ports;. Important changes to the way ports work will be announced
there. You can always find more detailed information on the latest
changes by looking at the
bsd.port.mk CVS log.That is It, Folks!Boy, this sure was a long tutorial, wasn't it? Thanks for
following us to here, really. Now that you know how to do a port,
have at it and convert everything in the world into ports! That
is the easiest way to start contributing to the FreeBSD Project!
:-)