This article discusses the different build mechanisms and relevant variables that affect the release of build artifacts.
Details
Details
The article is available live at this URL for easier reading:
https://fdp.bofh.network/en/articles/reproducible-build/
Diff Detail
Diff Detail
- Repository
- R9 FreeBSD doc repository
- Lint
Lint Skipped - Unit
Tests Skipped
Event Timeline
| documentation/content/en/articles/reproducible-build/_index.adoc | ||
|---|---|---|
| 33 | Those variables seem not being used in this article? | |
| documentation/content/en/articles/reproducible-build/_index.adoc | ||
|---|---|---|
| 80 | They're not necessarily public domain, they're just publicly released. | |
| 93 | ||
| 107 | I wouldn't say secteam is part of the re process at all; and security patches and integration into release branches is after the release is out not "during" the RE process | |
| 108 | Check them against what? | |
| 186 | ||
| 202 | maybe semi-automatically? | |
| 204 | biweekly? | |
| 248 | We probably need to clarify the targets in release/Makefile vs. the release/release.sh script and thermite here; e.g. TARGET and TARGET_ARCH are for the first one | |
| 278 | parallel builds are possible with or without MAKEOBJDIRPREFIX | |
| 282 | Probably choose an example that isn't the default | |
| 287 | These are not release artifact targets. I'm not sure there's a reason to document DESTDIR for release artifacts? | |
| 297 | Not sure these belong with release artifact builds either? | |
| 308 | And same here. Probably all of these belong in a different document? | |
| 357 | NO_PROFILE is an error (see 20200212 UPDATING) | |
| 360 | ||
| 360–361 | The double/triple negative here is confusing: release builds disable the feature that disables the logic that strips build timestamps it's probably better to explain that historically some build artifacts included build timestamps, user information etc. and that has now been removed by default. WITHOUT_REPRODUCIBLE_BUILD restores those non-reproducible things | |
| 362 | Note that for ISO and memstick release artifacts this is an internal detail between release/Makefile's targets and installworld et. al. The user doesn't have to set this - it is explicitly passed to the make invocations (distributeworld, packageworld, distributekernel, packagekernel, installworld, installkernel, distribution) in release/Makefile. | |
| 373 | This shouldn't be the case - any META_MODE-induced non-reproducibility is a bug to fix. Are there actual examples of this? | |
| 419 | is it fair to say that make release is the make target to build all release artifacts for one given TARGET/TARGET_ARCH pair? We should also have an example showing this or at least explicitly point out that this is a target in release/Makefile not the top-level | |
| 431 | ? | |
| 441 | something like "for each TARGET/TARGET_ARCH pair, release.sh:"? | |
| 508–509 | At least prior to 15.0. | |
| 555 | ||
| 561 | base.txz? | |
Comment Actions
I like this article. A lot. No specific feedback, just a general comment - who is the target audience for this?
I had 2 potential groups in mind:
- FreeBSD user/organisation who wants to build from sources, replicating the official release process as close as possible
- FreeBSD project itself, serving as an overall guide on what this process is and who is involved
For the former, I would make sure there is a clear "golden path" through the
myriad options and knobs.
For the latter, again we should also be clear what are the settings we use today are.
But the document might be more clear if it targeted just one group rather than both.
| documentation/content/en/articles/reproducible-build/_index.adoc | ||
|---|---|---|
| 107 | I had a discussion with @cperciva and he mentioned that secteam@ actually generates related freebsd-update deltas for the BET and RC releases and users can actually update to these intermediate releases either for testing or whatever the purpose is. So I have updated the description to match to that statement. | |
| 108 | See above comments. This line has also been updated to address this. | |
| 373 | This is something I will need to recheck. I am more than confirmed that I have faced this issue. | |
Comment Actions
Unless there are major objections we could commit this and tinker later. LGTM, thanks moin!
Comment Actions
No. We have made some plan changes during our last meeting. We want to add the entire release engineering process including how a downstream provider can actually build a custom image. At the moment I am still occupied with the final touch of the 14.3 Release Notes. And plan on reshuffling this after that.
| documentation/content/en/articles/reproducible-build/_index.adoc | ||
|---|---|---|
| 95 | Perhaps add: In many cases this work is done by FreeBSD developer who are also developers of the upstream projects. | |
| 147 | s/releng@/re@/ (here and elsewhere) | |
| 149 | Mismatched punctuation; if you use a dash after "teams" you should have one after "clusteradm". | |
| 158 | s/ in coordination with clusteradm//, there's coordination about the pkg repositories but not about the final release artifacts. | |
| 175 | s/roadmap/roadmap (typically in September of every odd-numbered year)/ | |
| 180 | s/a code slush/a 2 week code slush/ | |
| 182 | Since you mentioned regressions earlier, perhaps add "Any regressions which landed in the stable branch since the previous release will ideally be fixed by this point." | |
| 186 | Process has changed. Replace this with: "During this phase, developers must request that the Release Engineering Team merge any changes from stable/XX into the releng/XX.Y branch." | |
| 191 | I'd change this to say "security advisories and errata notices", since doing a X.Y.Z-RELEASE is extremely rare. | |
| 201 | s/Releases/Builds/ I don't want people to confuse the weekly snapshots with releases. | |
| 204 | s/fortnight/week/ | |
| 207 | Maybe also mention that they can be useful for tracking down when regressions were introduced. This is only possible using EC2 images (since we keep those forever while the weekly snapshots get deleted from download.freebsd.org after two weeks) but I've done it a few times. | |
| 211 | I usually call these minor releases. | |
| 227 | Major vs minor releases? | |
| 252 | Should probably link to https://www.freebsd.org/platforms/ somewhere in here. | |
| 336 | Include "boot loader" in the list here since that's a common point of confusion. | |
| 372 | "On most architectures, the default configuration file is GENERIC, but custom configurations can be specified through the KERNCONF variable." Since on powerpc64 it's GENERIC64. | |
| 418 | Internally, `make release`: * Runs `installworld`, `installkernel`, and `distribution` or `packages` into a release-specific layout. since the word "runs" is not relevant to the other bullet points. | |
| 469 | s/a separate/an extra/ since Thermite calls into release.sh. | |
| 480 | s/PkgBase/Pkgbase/ No camels here. | |
| 536 | There are also "set" packages such as FreeBSD-set-base which can be used to quickly select a useful collection of packages. | |