Page MenuHomeFreeBSD

build/development.7: System building examples
ClosedPublic

Authored by ziaee on Jan 26 2025, 7:00 PM.
Tags
None
Referenced Files
Unknown Object (File)
Mon, Oct 13, 5:34 AM
Unknown Object (File)
Sun, Oct 12, 5:07 PM
Unknown Object (File)
Sun, Oct 12, 4:52 PM
Unknown Object (File)
Sun, Oct 12, 2:57 PM
Unknown Object (File)
Sun, Oct 12, 12:25 PM
Unknown Object (File)
Thu, Oct 9, 8:29 PM
Unknown Object (File)
Thu, Oct 9, 8:29 PM
Unknown Object (File)
Thu, Oct 9, 8:28 PM

Details

Summary
build/development.7: System building examples

Consolidate examples to build the system in the build(7) manual where
the details about the syntax involved is documented. Remove them from
development(7), instead adding examples for patching the trees.

Reduce documentation proliferation and create consistent, maintainable
doc about building the system, eventually removing the COMMMON ITEMS
section of UPDATING so there is a single source of truth in the cannon
and well-integrated location, the in-band system reference manual.

This commit refactors the introduction of build(7) for presentation and
introduces six detailed examples, mostly from development(7) and
UPDATING, for building the system.

+ Example 1: Build and upgrade system in place
+ Example 2: Build and upgrade a custom kernel in place
+ Example 3: Build and upgrade a single piece of userspace
+ Example 4: Build and upgrade a loadable kernel module
+ Example 5: Quickly rebuild a kernel in place
+ Example 6: Cross-compiling for different architectures

MFC after:              3 days
Discussed with:         emaste, imp, jhb, ivy
Differential Revision:  https://reviews.freebsd.org/D48693
Test Plan
# apply patch
sh src/tools/tools/git/git-arc.sh patch D48693
# look at rendered manual on various consoles, consider split screen with existing
MANWIDTH=80 man share/man/man7/development.7
MANWIDTH=59 man share/man/man7/development.7
# evaluate for accuracy, consistency, location, flow, style, tenseness, omission
# test examples
# lint manual
mandoc -Tlint share/man/man 7/development.7

(I'm trying to flesh manage testing procedure out to eventually become a doc or something)

Diff Detail

Repository
rG FreeBSD src repository
Lint
Lint Not Applicable
Unit
Tests Not Applicable

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes

Switch to INSTKERNNAME, switch "KODIR and INSTKERNNAME" to or instead of
and, thanks @jhb! Still reflecting on what to do with LOCAL_MODULES.

We should discourage running git and building as root. I've commented on a few of those cases inline but leave this here as a general comment. There's a lot of existing examples/documentation out there that do so so it's going to be an ongoing process to update. We should at least split the build and install steps even if both are invoked as root now -- so $ make buildkernel followed by #make installkernel or $ make clean all followed by # make install and not # make kernel or # make all install clean.

share/man/man7/build.7
616

Agreed

INSTKERNNAME is used by Makefile.inc1 for make installkernel. It passes KERNEL=${INSTKERNNAME} to the submake invocation. It defaults to kernel

KODIR is used by sys/conf/kern.pre.mk and sys/conf/kern.post.mk, and defaults to /boot/${KERNEL}.

If we're going to include both of these we should provide advice on which one is preferred, but my suggestion would be to include only INSTKERNNAME.

970
978–980

IMO echo "PORTS_MODULES+=graphics/drm-kmod emulators/virtualbox-ose-kmod" >> /etc/src.conf is nicer; I'd avoid the heredoc unless we expect to make this multiple lines.

996

We should discourage running git and building as root. A simple way to do this with our existing processes/documentation would be to chown user:user /usr/src first, before carrying on with the existing steps. This can be a follow-on to these changes.

998

And if we build as an unprivileged user, as we should, this necessarily splits into make buildkernel as the user and make installkernel as root.

1037

Should again split buildkernel and installkernel.

1041

I wouldn't say this is "Alternatively" -- KERNCONF and INSTKERNNAME are orthogonal. You can set KERNCONF and install it into the standard location or a custom name, and you can install a GENERIC kernel into the standard location or a custom name.

1047
1056

Will want to split this as well,

$ make clean all
# make install
1080

Sometimes only one of TARGET/TARGET_ARCH is needed, but sometimes both -- depending on the target. We might want to include both in the example. We shouldn't use TARGET in some examples and TARGET_ARCH in others.

share/man/man7/development.7
78–81

Is there a reason cherry-picked should move to a new line? And similar rewrapping elsewhere.

138–159

Yes, git apply is appropriate for existing patches floating around, but if someone is submitting a new patch we should strongly prefer (or require) git am format.

ziaee marked 2 inline comments as done.

+ fix INSTKERNNAME example straggler
+ switch cross compiling examples to TARGET, I don't know if this works, these examples are from other docs we have
+ add an example for git formatted patches to development(7)

Attempt to improve language for INSTKERNNAME example

Oops, lots of pending messages didn't get submitted. Still reflecting on LOCAL_MODULES and not building as root. Do we have any mechanism to do this in base? mdo?

share/man/man7/build.7
616

This is much better, thank you.

616

One use of manual pages is to consult when reading stuff you don't understand. I don't want to remove this, if this is where it goes, since it is a real make variable for the kernel. Maybe we can add a note discouraging it?

956

Moved the note about manual intervention to the top of the examples section.

978–980

That is 92 characters including the offset, but writing it with a here doc is legible on standard console.

1010

What's wrong with reboot? I'm under the impression that poweroff and reboot trigger the correct sequence.

1047

Oops, that one was a straggler.

1080

It isn't clear to me when to use one or the other, these examples were existing elsewhere in the documentation. Included both for all examples in next push.

share/man/man7/development.7
78–81

I've been breaking long lines for legibility with vi on standard console.

138

I never understood the point of this package. It helps them work on the src tree, but the src tree already includes them.

138–159

I didn't know about that. I use git apply for every type of patch users submit on bugzilla and it always works. Is this the patches made with git format-patch?

share/man/man7/build.7
1080

included both for all examples in the next push

This was actually not true, I switched them to TARGET because the doc above says you can usually do that.

Split make kernel up for future recommendations about not building as
root.

looks good overall

share/man/man7/build.7
999

The build targets could be kept in one make invocation, i.e.,

make buildworld buildkernel
make installkernel

That's the way I do it. I think either is fine though, and we should choose whichever is more clear/understandable for new developers.

1080

I'm not sure which should be preferred, maybe @jhb or @imp has insight.

The mapping is handled by Makefile:

# Guess target architecture from target type, and vice versa, based on
# historic FreeBSD practice of tending to have TARGET == TARGET_ARCH
# expanding to TARGET == TARGET_CPUARCH in recent times, with known
# exceptions.
.if !defined(TARGET_ARCH) && defined(TARGET)
# T->TA mapping is usually TARGET with arm64 the odd man out
_TARGET_ARCH=   ${TARGET:S/arm64/aarch64/:S/riscv/riscv64/:S/arm/armv7/}
.elif !defined(TARGET) && defined(TARGET_ARCH) && \
    ${TARGET_ARCH} != ${MACHINE_ARCH}
# TA->T mapping is accidentally CPUARCH with aarch64 the odd man out
_TARGET=        ${TARGET_ARCH:${__TO_CPUARCH}:C/aarch64/arm64/}
.endif
share/man/man7/development.7
78–81

but the old line is only 73 characters so it seems like unnecessary churn/review friction, unless we want to introduce mdoc style that suggests a 72 character maximum or so?

share/man/man7/build.7
1010

traditionally, reboot(8) does not run rc.d shutdown (stop) scripts, you are supposed to use shutdown -r to do that. there's been some discussion about changing this but as far as i know it hasn't happened yet.

confusing, poweroff(8) does invoke shutdown, which seems like a poor choice in hindsight.

share/man/man7/development.7
138

some people prefer to install things from packages instead of manually (or using the Makefile) because a) it means you can find out where the thing came from later, and b) you don't have to remember to keep updating it by hand.

138–159

yes, this is for git format-patch. you can usually apply these with git apply but you will lose the Git metadata: author, commit message, etc.

ziaee marked 4 inline comments as done.

+ combine build targets in examples
+ markup buildworld in CAVEATS
+ add a CAVEAT about build. The way I see this going is that the already
architecture specific boot(8) pages (missing on ARM64, and a bit
convoluted on x86) needs to be the meta page or topic introduction page
for each architecture, linking to the loader pages for that arch.
We just have too many pages all over the place for this with great doc
sprawl. It makes sense to start with boot(8) and clean them up from
there but like, that needs a whole working group or I need to annoy
everyone with questions.

ziaee edited the test plan for this revision. (Show Details)
share/man/man7/build.7
1010

Yes, reboot is basically never what you want.

1140–1145

We should have both better tooling and better documentation for bootloader updates but just calling it out here is a good improvement. Some folks are not aware that the bootloader does not get updated automatically.

Switch all reboot to shutdown -r now, and remove xref to reboot.

reboot is not what you want

That's depressing. I would be in favor of poweroff and reboot being
sane commands, since they're human friendly, if we can do it without
breaking things. It's obvious halt is almost never what you want, it
has a very sane name, to me. TIL, thank you both.

share/man/man7/build.7
999

we should choose whichever is more clear/understandable for new developers.

+1 <3

1080

What would be really nice if possible, like we did with ls, is clean up the logic instead of describing extra complexity. I.e. if it's possible that the above doc could be made more absolute that TARGET always sets TARGET_ARCH. This would be elegant.

share/man/man7/development.7
78–81

we want to introduce mdoc style that suggests a 72 character maximum or so?

This would make me very, very happy. I use 80 columns and vi, leaving 72 columns, and I feel this has been the standard since generations, and also is the BSD native tooling in base. Also there was an RFC about wrapping at 72. Also I have made some memes to this effect. But I feel that I do not yet have enough reputation to present a purely style style rule.

share/man/man7/build.7
945

Split out the mechanical from the content changes..

1080

The rules are like so: TARGET_ARCH turns out to have a unique mapping to TARGET. All our current platforms it's unique. Since we retired pc98, we no longer have any ambiguity. TARGET is likewise usually sufficient, except for powerpc. powerpc has multiple TARGET_ARCH that it could be. In that case, you have to specify both.

We've been weakly encouraging TARGET_ARCH here for a while. It also works for the CHERI folks. TARGET works for all the 'mainstream' systems , but not powerpc. However, powerpc is, relatively speaking, fairly niche. But it's easier just to encourage TARGET_ARCH which avoids a lot of this complication. Generally, only kernel hackers need to know about TARGET since it corresponds to MACHINE which is where all the MD files for a kernel reside (src/sys/$MACHINE).

+ rollback fixing list widths
+ switch to TARGET_ARCH=aarch64 in examples, thanks @imp!

share/man/man7/build.7
1080

So the far above comment that "cross-building for ARM64 machines requires TARGET_ARCH... and TARGET..." is no longer true or should be updated?

ziaee marked 4 inline comments as done.

+ split make clean all install
+ make cross-building example DESTDIR consistent

ziaee marked 5 inline comments as done and 3 inline comments as done.

Development(7):
+ markup draft.diff/draft.patch as paths, for e.g. apropos Pa=patch
+ add the port of git-arc to FILES, and then xref it's manual

For another review:
+ we should move git-arc(1) into the default install. We do that already
with something else from src/tools/, I don't remember now what it is.

ziaee marked 4 inline comments as done.May 9 2025, 6:42 PM
ziaee added inline comments.
share/man/man7/build.7
1140–1145

Strong agree. This is a whole thing, and it needs to be mentioned on a per-loader basis in EXAMPLES in their respective manuals.

share/man/man7/development.7
138

Okay, I looked for a the least clumsy way to do that. Note that both portmgr and doc historically (including yesterday) objects to ports being mentioned in manuals, but I don't see because what else we could really do.

ziaee marked an inline comment as done.

Add a DIAGNOSTICS section showing the Bad system call error.

share/man/man7/build.7
1125

Note that it will also emit rescue/sh check failed, installation aborted -- ideally we could list both lines of output.

Also maybe include a note e.g. Incorrect build procedure (kernel not updated)?

Awesome. I elected to use the formatting in gve for multiline errors.
Thanks @emaste!

fix alphabetization of added ENVIRONMENT/KODIR entry

Might be a few more nits, but we're getting close to time to just accept what's done and do further refinement later.

share/man/man7/build.7
968

Or better yet, use boot environments.

1128

cleandepend is better for the second one of these. the two cleandir target trick is quite obsolete by now I thought.

This revision is now accepted and ready to land.Fri, Sep 19, 2:01 AM
This revision was automatically updated to reflect the committed changes.