Changeset View
Standalone View
en_US.ISO8859-1/books/porters-handbook/ph-build-process/chapter.xml
<?xml version="1.0" encoding="iso-8859-1"?> | |||||
<!-- | |||||
The FreeBSD Documentation Project | |||||
$FreeBSD: head/en_US.ISO8859-1/books/porters-handbook/slow-porting/chapter.xml 50801 2017-09-04 14:50:49Z mat $ | |||||
--> | |||||
<chapter xmlns="http://docbook.org/ns/docbook" | |||||
xmlns:xlink="http://www.w3.org/1999/xlink" version="5.0" | |||||
xml:id="ph-build-process"> | |||||
<title>Build Process and Port Contents</title> | |||||
<sect1 xml:id="build-overview"> | |||||
<title>How FreeBSD Makes Ports</title> | |||||
<para> | |||||
The FreeBSD ports system uses the make(?) build tool and contains pre-created "build targets" for most of the things that you will need for a given port. Build targets are instructions that tell make(?) how to perform tasks. These tasks include downloading sources, building the software and installation. Build targets for the ports system live under the ports tree under mk/. These files also reference the base FreeBSD make files at ?. For those curious about how the specific targets work, the full documentation is available in the source code. Please refer to the <filename>bsd.port.mk</filename> file as it is well commented. | |||||
</para> | |||||
mat: s/FreeBSD/&os;/
s/the make(?) build tool/&man.make.1;/
Never use "you" or "your"
`the ports… | |||||
Not Done Inline ActionsThe &os; ports system utilizes the &man.make.1; build tool to create usable instances of software. Ports contains pre-created "build targets" for most processes required to build software and is extensible to meet any unforseen need. Build targets are instructions that tell &man.make.1; how to perform tasks such as downloading sources, building the software and installation. Build targets for the ports system live under the ports tree under <filename>Mk/</filename>. russ.haley_gmail.com: The &os; ports system utilizes the &man.make.1; build tool to create usable instances of… | |||||
<note> | |||||
<para> | |||||
Sage advice from the previous author: | |||||
</para> | |||||
<para>But do not worry, not many people understand exactly how | |||||
<filename>bsd.port.mk</filename> is working... | |||||
<!-- smiley --><emphasis>:-)</emphasis> | |||||
</para> | |||||
</note> | |||||
<para> | |||||
First, let us describe the sequence of events which occurs when the | |||||
user types <command>make</command> for the first time in a given port directory. | |||||
</para> | |||||
<procedure> | |||||
<step> | |||||
<para>The <buildtarget>fetch</buildtarget> target is run. The | |||||
<buildtarget>fetch</buildtarget> target is responsible for | |||||
making sure that the tarball exists locally in | |||||
<varname>DISTDIR</varname>. If | |||||
<buildtarget>fetch</buildtarget> cannot find the required | |||||
files in <varname>DISTDIR</varname> it will look up the URL | |||||
<varname>MASTER_SITES</varname>, which is set in the | |||||
Makefile, as well as our FTP mirrors where we put distfiles | |||||
as backup. It will then attempt to fetch the named | |||||
distribution file with <varname>FETCH</varname>, assuming | |||||
that the requesting site has direct access to the Internet. | |||||
If that succeeds, it will save the file in | |||||
<varname>DISTDIR</varname> for future use and | |||||
proceed.</para> | |||||
</step> | |||||
<step> | |||||
<para>The <buildtarget>extract</buildtarget> target is run. | |||||
It looks for the port's distribution file (typically a | |||||
<command>gzip</command>ped tarball) in | |||||
<varname>DISTDIR</varname> and unpacks it into a temporary | |||||
Not Done Inline Actions compressped tarball) in mat: ` compressped tarball) in` | |||||
Not Done Inline Actions(typically a compressed archive) russ.haley_gmail.com: (typically a compressed archive) | |||||
subdirectory specified by <varname>WRKDIR</varname> | |||||
Not Done Inline ActionsYou could add that it checks the file integrity through a checksum. mat: You could add that it checks the file integrity through a checksum. | |||||
(defaults to <filename>work</filename>).</para> | |||||
</step> | |||||
Not Done Inline ActionsDon't say that, because THIS MUST NOT BE CHANGED from its default. mat: Don't say that, because **THIS MUST NOT BE CHANGED** from its default. | |||||
Not Done Inline ActionsAlso, it is no longer true with flavors. mat: Also, it is no longer true with flavors. | |||||
Not Done Inline Actions<step> <para>The <buildtarget>extract</buildtarget> target is run. The system looks for the port's distribution file (typically a compressed archive) in <varname>DISTDIR</varname> and unpacks it into a temporary subdirectory specified by <varname>WRKDIR</varname> (resolves to <filename>work</filename> under the top port directory). File integrity is verified by checksuming the download and comparing to the checksum value specifified in the <filename>distinfo</filename> file. </para> </step> russ.haley_gmail.com: <step>
<para>The <buildtarget>extract</buildtarget> target is run.
The system looks… | |||||
<step> | |||||
<para>The <buildtarget>patch</buildtarget> target is run. | |||||
First, any patches defined in <varname>PATCHFILES</varname> | |||||
are applied. Second, if any patch files named | |||||
<filename>patch-<replaceable>*</replaceable></filename> are | |||||
found in <varname>PATCHDIR</varname> (defaults to the | |||||
<filename>files</filename> subdirectory), they are applied | |||||
at this time in alphabetical order.</para> | |||||
</step> | |||||
<step> | |||||
<para>The <buildtarget>configure</buildtarget> target is run. | |||||
This can do any one of many different things.</para> | |||||
Not Done Inline ActionsThe optional configure target is run. mat: The optional configure target is run. | |||||
Not Done Inline Actionsremoved as per next comment russ.haley_gmail.com: removed as per next comment | |||||
<orderedlist> | |||||
<listitem> | |||||
<para>If it exists, <filename>scripts/configure</filename> | |||||
is run.</para> | |||||
</listitem> | |||||
Done Inline ActionsI would really like you not to document this there are only a handful of very old crufty ports doing this, and they should be converted to work with a modern way of doing things. mat: I would really like you not to document this there are only a handful of very old crufty ports… | |||||
<listitem> | |||||
<para>If <varname>HAS_CONFIGURE</varname> or | |||||
<varname>GNU_CONFIGURE</varname> is set, | |||||
<filename>WRKSRC/configure</filename> is run.</para> | |||||
</listitem> | |||||
</orderedlist> | |||||
</step> | |||||
<step> | |||||
<para>The <buildtarget>build</buildtarget> target is run. | |||||
This is responsible for descending into the port's private | |||||
working directory (<varname>WRKSRC</varname>) and building | |||||
it.</para> | |||||
</step> | |||||
<step> | |||||
<para>The <buildtarget>stage</buildtarget> target is run. | |||||
This puts the final set of built files into a temporary | |||||
directory (<varname>STAGEDIR</varname>, see | |||||
Done Inline ActionsThis installs the final set of built files into a staging mat: This installs the final set of built files into a staging
| |||||
<xref linkend="staging"/>). The hierarchy of this directory | |||||
mirrors that of the system on which the package will be | |||||
installed.</para> | |||||
</step> | |||||
<step> | |||||
<para>The <buildtarget>package</buildtarget> target is run. | |||||
This creates a package using the files from the temporary | |||||
directory created during the | |||||
<buildtarget>stage</buildtarget> target and the port's | |||||
<filename>pkg-plist</filename>.</para> | |||||
</step> | |||||
<step> | |||||
<para>The <buildtarget>install</buildtarget> target is run. | |||||
This installs the package created during the | |||||
<buildtarget>package</buildtarget> target into the host | |||||
system. | |||||
</para> | |||||
</step> | |||||
</procedure> | |||||
Not Done Inline ActionsBoth of those step are not run when a user types make. mat: Both of those step are **not** run when a user types `make`. | |||||
Not Done Inline Actions"a package" is opaque. Can someone give me details on what it is? Is it compatible with pkg(8)? Can it be distributed? Also, does make install use "the package" or the files from the staging? russ.haley_gmail.com: "a package" is opaque. Can someone give me details on what it is? Is it compatible with pkg(8)? | |||||
<para>If more control over the build process is required, additional <buildtarget>pre-<replaceable>*</replaceable></buildtarget> and <buildtarget>post-<replaceable>*</replaceable></buildtarget> targets can be added to the process | |||||
either in the Makefile or by creating a script named as above in the <filename>scripts</filename> subdirectory. These targets/scripts will be run prior-to or subsequently-to the named target as the prefix implies. | |||||
</para> | |||||
Done Inline Actionssame here, don't talk about the scripts. (this goes for the whole document.) mat: same here, don't talk about the scripts. (this goes for the whole document.) | |||||
<para>For example, if there is a | |||||
<buildtarget>post-extract</buildtarget> target defined in the | |||||
<filename>Makefile</filename>, and a file | |||||
<filename>pre-build</filename> in the | |||||
<filename>scripts</filename> subdirectory, the | |||||
<buildtarget>post-extract</buildtarget> target will be called | |||||
after the regular extraction actions, and | |||||
<filename>pre-build</filename> will be executed before | |||||
the default build rules are done. It is recommended to | |||||
use <filename>Makefile</filename> targets if the actions are | |||||
simple enough, because it will be easier for someone to figure | |||||
out what kind of non-default action the port requires. | |||||
</para> | |||||
<para>The default actions are done by the | |||||
<buildtarget>do-<replaceable>something</replaceable></buildtarget> | |||||
targets from <filename>bsd.port.mk</filename>. | |||||
For example, the commands to extract a port are in the target | |||||
<buildtarget>do-extract</buildtarget>. If | |||||
the default target does not do the job right, redefine the | |||||
<buildtarget>do-<replaceable>something</replaceable></buildtarget> | |||||
target in the <filename>Makefile</filename>. | |||||
</para> | |||||
<note> | |||||
<para> | |||||
*Authors Note: Should we really be talking about modifying bsd.ports.mk here? | |||||
</para> | |||||
Not Done Inline ActionsWe should never talk about modifying bsd.port.mk or even tell people to go and read it. All the documentation people need should be here, if they need to go read b.p.m we failed. mat: We should never talk about modifying bsd.port.mk or even tell people to go and read it.
All… | |||||
Not Done Inline ActionsI've removed all references to the scripts. However, as a developer I humbly disagree about the need to read source code. At a certain point the vernacular only obfuscates what high level languages have been designed to express. I have re-written the final paragraph to remove comments about the sources, but would like to leave the preceding comment directed "at the curious". russ.haley_gmail.com: I've removed all references to the scripts. However, as a developer I humbly disagree about the… | |||||
Not Done Inline ActionsHi mat, the following link to the documentation primer is perhaps a precedent for referring to source code in documentation? I knew I had seen it before, but was unsure where to find it. https://www.freebsd.org/doc/en_US.ISO8859-1/books/fdp-primer/make-includes.html Thoughts? russ.haley_gmail.com: Hi mat, the following link to the documentation primer is perhaps a precedent for referring to… | |||||
</note> | |||||
<note> | |||||
<para> | |||||
Never modify the <quote>main</quote> targets (for example, | |||||
<buildtarget>extract</buildtarget>, | |||||
<buildtarget>configure</buildtarget>, etc.). These targets are groupings | |||||
that make sure all prior stages are completed and then | |||||
call the real targets or scripts. If there is an issue in the extraction process, fix | |||||
<buildtarget>do-extract</buildtarget>, but never ever change | |||||
the way <buildtarget>extract</buildtarget> operates! | |||||
Additionally, the target | |||||
<buildtarget>post-deinstall</buildtarget> is invalid and is | |||||
not run by the ports infrastructure. | |||||
</para> | |||||
Done Inline ActionsSo don't talk about it :-) mat: So don't talk about it :-) | |||||
</note> | |||||
<para> | |||||
Now that we have an understanding of what happens when the user executes <command>make | |||||
install</command>, let us go through the | |||||
recommended steps to create the perfect port. | |||||
</para> | |||||
</sect1> | |||||
<sect1 xml:id="ph-basic-files"> | |||||
<title>Basic port structure</title> | |||||
<para> | |||||
A port is a directory of source files that contains all the information the ports build system needs to generate a new executable on FreeBSD. The following sections describe a typical port containing a package description, the list of files to be installed, a checksum file to validate the items downloaded, a directory of patch files, and of course, a make file. | |||||
</para> | |||||
<sect2 xml:id="ph-description-file"> | |||||
<title><filename>pkg-descr</filename></title> | |||||
Done Inline Actionsthere is no such thing a "checksum". Don't document the files directory, it is optional. mat: there is no such thing a "checksum". Don't document the files directory, it is optional. | |||||
Not Done Inline ActionsOops on the checksum. I don't understand why the files directorys' optional status makes it un-worthy of documentation, especially given your prior statement about this containing all information required for ports? Is the directory not required for all patch files? russ.haley_gmail.com: Oops on the checksum. I don't understand why the files directorys' optional status makes it un… | |||||
<para><filename>pkg-descr</filename> contains a description of the port. The file should concisely explaining what the port does.It should be at least one paragraph, but should not be more than four or five paragraphs. </para> | |||||
<note> | |||||
<para>This is <emphasis>not</emphasis> a manual or an | |||||
in-depth description on how to use or compile the port! | |||||
<emphasis>Please be careful when copying from the | |||||
<filename>README</filename> or manpage</emphasis>. Too | |||||
often they are not a concise description of the port or | |||||
are in an awkward format. For example, manpages have | |||||
justified spacing, which looks particularly bad with | |||||
monospaced fonts.</para> | |||||
<para>On the other hand, the content of | |||||
<filename>pkg-descr</filename> must be longer than the <link | |||||
linkend="makefile-comment"><varname>COMMENT</varname></link> | |||||
line from the Makefile. It must explain in more depth what | |||||
the port is all about.</para> | |||||
</note> | |||||
<para>A well-written <filename>pkg-descr</filename> describes | |||||
Not Done Inline ActionsThere is already a section about this in the "quick porting" chapter, it should probably be enhanced rather than duplicate things here. mat: There is already a section about this in the "quick porting" chapter, it should probably be… | |||||
Not Done Inline ActionsSorry, I wasn't clear enough in my description. My intent is to remove porting-why, quick-porting, and slow porting. All the information in those chapters has been subsumed in the new chapters. russ.haley_gmail.com: Sorry, I wasn't clear enough in my description. My intent is to remove porting-why, quick… | |||||
Not Done Inline ActionsMmmm, I do not think this is a great idea, if the point is just to move things around, then move them around in the existing chapters. mat: Mmmm, I do not think this is a great idea, if the point is just to move things around, then… | |||||
Not Done Inline ActionsIn the end I am indifferent to the chapter names. However (and again, with all due respect and appreciation), I think you can agree I have done much more than just moved things around. Also, I fully expect to flesh out or otherwise bring the original content in line with feedback, which has already had substantial changes (to said original text). Given the above, I think it would be fair to say that the original chapter names are no longer applicable and - in my opinion - were of dubious value at best. Can you provide some reasoning as to why you wish to see the original chapter names preserved? russ.haley_gmail.com: In the end I am indifferent to the chapter names. However (and again, with all due respect and… | |||||
Not Done Inline ActionsWell, the main problem I see is that unless you copy everything and then modify (which is about the same as changing in place), bits and pieces of documentation will be lost, which I find unacceptable. mat: Well, the main problem I see is that unless you copy everything and then modify (which is about… | |||||
Not Done Inline ActionsOf course, quite right. I shall create a new checkout to get it to build first, and then impose the new content on the old chapters for comparison purposes. I will note a request to rename in the future? russ.haley_gmail.com: Of course, quite right. I shall create a new checkout to get it to build first, and then impose… | |||||
Not Done Inline ActionsHi Mat, I just wanted to say I haven't forgotten about this comment. I've aproached it a few times but with no satisfactory answers. The content is really a complete re-write. I can put together a listing of the source of the content, but there is no real way to diff the chapters. I'm happy to put that together if you wish, but a far more productive aproach (in my opinion) would be proper community vetting. Thoughts? russ.haley_gmail.com: Hi Mat, I just wanted to say I haven't forgotten about this comment. I've aproached it a few… | |||||
the port completely enough that users would not have to | |||||
consult the documentation or visit the website to understand | |||||
what the software does, how it can be useful, or what | |||||
particularly nice features it has. Mentioning certain | |||||
requirements like a graphical toolkit, heavy dependencies, | |||||
runtime environment, or implementation languages help users | |||||
decide whether this port will work for them.</para> | |||||
<para>Include a URL to the official WWW homepage. Prepend | |||||
<emphasis>one</emphasis> of the websites (pick the most | |||||
common one) with <literal>WWW:</literal> (followed by single | |||||
space) so that automated tools will work correctly. If the | |||||
URI is the root of the website or directory, it must be | |||||
terminated with a slash.</para> | |||||
<note> | |||||
<para>If the listed webpage for a port is not available, try | |||||
to search the Internet first to see if the official site | |||||
moved, was renamed, or is hosted elsewhere.</para> | |||||
</note> | |||||
<para>This example shows how | |||||
<filename>pkg-descr</filename> looks:</para> | |||||
<programlisting>This is a port of oneko, in which a cat chases a poor mouse all over | |||||
the screen. | |||||
: | |||||
(etc.) | |||||
WWW: http://www.oneko.org/</programlisting> | |||||
</sect2> | |||||
<sect2 xml:id="ph-plist-file"> | |||||
<title><filename>pkg-plist</filename></title> | |||||
<para>This file lists all the files installed by the port. It | |||||
is also called the <quote>packing list</quote> because the | |||||
package is generated by packing the files listed here. The | |||||
pathnames are relative to the installation prefix (usually | |||||
<filename>/usr/local</filename>).</para> | |||||
<para>Here is a small example:</para> | |||||
<programlisting>bin/oneko | |||||
man/man1/oneko.1.gz | |||||
lib/X11/app-defaults/Oneko | |||||
lib/X11/oneko/cat1.xpm | |||||
lib/X11/oneko/cat2.xpm | |||||
lib/X11/oneko/mouse.xpm</programlisting> | |||||
<para>Refer to the &man.pkg-create.8; manual page for details | |||||
on the packing list.</para> | |||||
<note> | |||||
<para>It is recommended to keep all the filenames in | |||||
this file sorted alphabetically. It will make verifying | |||||
changes when upgrading the port much easier.</para> | |||||
</note> | |||||
<tip> | |||||
<para>Creating a packing list manually can be a very tedious | |||||
task. If the port installs a large numbers of files, | |||||
<link linkend="plist-autoplist">creating the packing list | |||||
automatically</link> might save time.</para> | |||||
</tip> | |||||
<para>There is only one case when | |||||
<filename>pkg-plist</filename> can be omitted from a port. | |||||
If the port installs just a handful of files, list them in | |||||
<varname>PLIST_FILES</varname>, within the | |||||
port's <filename>Makefile</filename>. For instance, we | |||||
could get along without <filename>pkg-plist</filename> in | |||||
the above <filename>oneko</filename> port by adding these | |||||
lines to the <filename>Makefile</filename>:</para> | |||||
<programlisting>PLIST_FILES= bin/oneko \ | |||||
man/man1/oneko.1.gz \ | |||||
lib/X11/app-defaults/Oneko \ | |||||
lib/X11/oneko/cat1.xpm \ | |||||
lib/X11/oneko/cat2.xpm \ | |||||
lib/X11/oneko/mouse.xpm</programlisting> | |||||
<note> | |||||
<para>Usage of <varname>PLIST_FILES</varname> should not be | |||||
abused. When looking for the origin of a file, people | |||||
usually try to <application>grep</application> through the | |||||
<filename>pkg-plist</filename> files in the ports tree. | |||||
Listing files in <varname>PLIST_FILES</varname> in the | |||||
<filename>Makefile</filename> makes that search more | |||||
difficult.</para> | |||||
</note> | |||||
<tip> | |||||
<para>If a port needs to create an empty directory, or creates | |||||
directories outside of <filename>${PREFIX}</filename> during | |||||
installation, refer to <xref linkend="plist-dir-cleaning"/> | |||||
for more information.</para> | |||||
</tip> | |||||
<para>The price for this way of listing a port's files and | |||||
directories is that the keywords described in | |||||
&man.pkg-create.8; and <xref linkend="plist-keywords"/> cannot | |||||
be used. Therefore, it is suitable | |||||
only for simple ports and makes them even simpler. At the | |||||
same time, it has the advantage of reducing the number of | |||||
files in the ports collection. Please consider using this | |||||
technique before resorting to | |||||
<filename>pkg-plist</filename>.</para> | |||||
<para>Later we will see how <filename>pkg-plist</filename> | |||||
and <varname>PLIST_FILES</varname> can be used to fulfill | |||||
<link linkend="plist">more sophisticated | |||||
tasks</link>.</para> | |||||
</sect2> | |||||
<sect2 xml:id="ph-checksum-file"> | |||||
<title>The checksum File</title> | |||||
<para> | |||||
The checksum file contains a number based on a calculation of all the files within the port. This number is used as a tool to verify the contents to ensure the expected files are present. While not a perfect mechanism, it is accurate enough to fulfill the roll of validation but not necessarily tampering protection. Checksum files are created by running the make checksum command within the port folder. | |||||
</para> | |||||
</sect2> | |||||
<sect2 xml:id="ph-files-dir"> | |||||
<title>The Files directory</title> | |||||
<para> | |||||
The files directory contains the patch files used to update the original sources to allow the system to run on FreeBSD. There are other reasons to create and include patch files but for the sake of this section, "patching the original sources" is an adequate description. The files contained in this directory are "required" to be generated by the make patch target. However, this is a software requirement and manual patching is still an option. | |||||
</para> | |||||
</sect2> | |||||
<sect2 xml:id="ph-makefile"> | |||||
<title>The <filename>Makefile</filename></title> | |||||
<para> | |||||
The Makefile is the entrypoint for the ports system to build the binary package. This makefile is NOT the target application makefile. Rather, this makefile is a meta-package that contains the FreeBSD instructions outlined in the first section of this chapter. This file can contain very complex instructions, but typically includes variables for: | |||||
- The port name | |||||
- The port version | |||||
- Port "meta info" (information about the port) | |||||
Done Inline ActionsIt is called distinfo. mat: It is called distinfo. | |||||
- Master sources site | |||||
- Source code version to retrieve | |||||
- Additional build instructions | |||||
- Additional make targets | |||||
</para> | |||||
<para>The minimal <filename>Makefile</filename> would look | |||||
something like this:</para> | |||||
Done Inline Actions<title>The <filename>files</filename> directory</title> mat: <title>The <filename>files</filename> directory</title> | |||||
<programlisting># $FreeBSD$ | |||||
PORTNAME= oneko | |||||
DISTVERSION= 1.1b | |||||
CATEGORIES= games | |||||
MASTER_SITES= ftp://ftp.cs.columbia.edu/archives/X11R5/contrib/ | |||||
MAINTAINER= youremail@example.com | |||||
COMMENT= Cat chasing a mouse all over the screen | |||||
.include <bsd.port.mk></programlisting> | |||||
<note> | |||||
<para>In some cases, the <filename>Makefile</filename> of an | |||||
existing port may contain additional lines in the header, | |||||
such as the name of the port and the date it was created. | |||||
This additional information has been declared obsolete, and | |||||
is being phased out.</para> | |||||
</note> | |||||
<para>Try to figure it out. Do not worry about the | |||||
contents of the <literal>$FreeBSD$</literal> | |||||
line, it will be filled in automatically by | |||||
<application>Subversion</application> when the port is | |||||
imported to our main ports tree. A more detailed | |||||
example is shown in the | |||||
<link linkend="porting-samplem">sample Makefile</link> | |||||
section.</para> | |||||
<para> | |||||
This section is by no means a complete reference to porting. This is intended to give the reader enough information to understand what a port looks like and how the system works. Once again, it is recommended that those seeking more information on the build process and the file contents review the build target information in the bsd.ports.mk file. Now, let us look at the steps to maintain a port. | |||||
</para> | |||||
</sect2> | |||||
</sect1> | |||||
</chapter> |
s/FreeBSD/&os;/
s/the make(?) build tool/&man.make.1;/
Never use "you" or "your"
the ports tree under <filename>Mk/</filename>.