Page MenuHomeFreeBSD

makefs: Honor -T timestamps when creating images from mtree manifests
ClosedPublic

Authored by markj on Mar 27 2025, 1:47 PM.
Tags
None
Referenced Files
Unknown Object (File)
Thu, Oct 9, 7:10 AM
Unknown Object (File)
Tue, Oct 7, 4:48 AM
Unknown Object (File)
Sat, Oct 4, 12:49 PM
Unknown Object (File)
Fri, Oct 3, 6:08 AM
Unknown Object (File)
Thu, Oct 2, 12:29 PM
Unknown Object (File)
Thu, Oct 2, 2:52 AM
Unknown Object (File)
Sat, Sep 20, 5:06 PM
Unknown Object (File)
Wed, Sep 17, 11:20 AM

Details

Summary

makefs backends rely on the fsnode structure to derive most
of the information about the underlying filesystem objects.
Depending on how the image is built, the fsnode structures
are initialized in the walk_dir or read_mtree functions.
However, read_mtree fails to take timestamps passed by -T into account,
leading to nonreproducible images in backends that do not check for -T.
Fix this and make -T backend-agnostic by adding an appropriate check
in read_mtree_keywords while making sure that mtree entries can
still override -T timestamps.

PR: 285630
Sponsored by: Klara, Inc.
Sponsored by: The FreeBSD Foundation

Diff Detail

Repository
rG FreeBSD src repository
Lint
Lint Not Applicable
Unit
Tests Not Applicable

Event Timeline

This looks like a much nicer solution.

Does this mean we can remove references to stampst in the backends, e.g., ffs_build_dinode2()?

usr.sbin/makefs/mtree.c
726–728

Does this mean we can remove references to stampst in the backends, e.g., ffs_build_dinode2()?

Yes, I think ffs and msdos have stampst checks and I plan on removing them in a separate commit (along with adding appropriate tests for each backend).

We need to make sure that all four combinations work -- mtree mode / directory scan, -T / no -T

This revision is now accepted and ready to land.Mar 28 2025, 1:50 PM

There are three possible timestamps of interest:

  1. As stored in the filesystem
  2. Specified in a mtree file
  3. Overridden timestamp for reproducible builds (cmdline arg or SOURCE_DATE_EPOCH)

In this list higher numbered timestamps should take priority over lower ones.

Address @emaste 's comments - mtree timestamps will now no longer override the default timestamp passed by -T.

This revision now requires review to proceed.Apr 1 2025, 11:49 AM
kevans added inline comments.
usr.sbin/makefs/makefs.8
262

I would've perhaps instead changed this to note that they're explicitly ignored if a default timestamp is provided.

usr.sbin/makefs/makefs.8
262

Note that there are two distinct uses of mtree in makefs: the "normal" way, where the mtree provides the file metadata rather than the input file tree, and "specfile mode", where the mtree file acts as a filter on the input file tree. In specfile mode, input files that don't have a corresponding mtree entry are ignored, and mtree entries that don't have a corresponding input file (usually a directory) are synthesized from the mtree attributes. And, mtree attributes, such as timestamps, override those of the input filesystem.

For release building I believe we do not care about specfile mode. But the deleted comment is still correct, as apply_specdir() still ignores the -T timestamp (and this is probably the right behaviour). It would be good to further clarify how -T interacts with the two modes.

Clarify interaction between -F and -T.

usr.sbin/makefs/makefs.8
262

For release building I believe we do not care about specfile mode. But the deleted comment is still correct, as apply_specdir() still ignores the -T timestamp (and this is probably the right behaviour). It would be good to further clarify how -T interacts with the two modes.

Thanks for pointing this out, I've clarified the interaction between the two. I'll also add tests for testing this interaction.

usr.sbin/makefs/makefs.8
264–268

This is still a bit confusing IMO: "spec file" already refers to the mtree passed with -F. Otherwise the makefs documentation refers to it as a "manifest." I'd suggest something like the following, to be more explicit: "Timestamps in a mtree(5) specfile (specified with -F) are used even if a default timestamp is specified. However, the timestamps in an mtree manifest are ignored if a default timestamp is specified."

bnovkov marked an inline comment as done.

Address @markj 's comments.

This revision is now accepted and ready to land.Apr 11 2025, 1:50 PM
usr.sbin/makefs/mtree.c
1059–1068

This timestamp also needs to come from stampst. I'll update the review with that change shortly.

markj edited reviewers, added: bnovkov; removed: markj.
This revision now requires review to proceed.May 9 2025, 2:06 PM
  • Fix the man page.
  • Update mtree_global_inode as well.
  • Factor out some duplicated code.
This revision was not accepted when it landed; it landed in state Needs Review.May 20 2025, 9:51 AM
This revision was automatically updated to reflect the committed changes.