Page MenuHomeFreeBSD

tools: build: add a rewrite of makeman in lua
ClosedPublic

Authored by kevans on Mar 15 2023, 5:32 AM.
Tags
None
Referenced Files
Unknown Object (File)
Wed, Oct 8, 3:38 AM
Unknown Object (File)
Tue, Oct 7, 2:57 AM
Unknown Object (File)
Mon, Oct 6, 8:28 PM
Unknown Object (File)
Sun, Oct 5, 2:16 AM
Unknown Object (File)
Sat, Oct 4, 2:59 PM
Unknown Object (File)
Tue, Sep 30, 1:41 AM
Unknown Object (File)
Mon, Sep 29, 8:03 PM
Unknown Object (File)
Mon, Sep 22, 12:43 AM

Details

Summary

The primary benefit of this rewrite is that it parallelizes a number of
the make(1) jobs that it needs to do. It does so with a very naive
forking model that could likely be improved, but is sufficient for our
purposes. This version also doesn't assume that CWD is sane, and
instead operates relative to the directory the script resides in.

Note that this initial version is only intended to match the output of
the legacy script. Some work is planned afterward to refactor the
script out into various components to improve maintainability after we
have switched over to it.

In my horribly performing dev environment, this version runs in 40s
rather than the original ~2 minutes. On a Mt. Snow machine, this
version runs in ~15s rather than the original ~1m40s.

Diff Detail

Repository
rG FreeBSD src repository
Lint
Lint Skipped
Unit
Tests Skipped
Build Status
Buildable 64428
Build 61312: arc lint + arc unit

Event Timeline

bapt added a subscriber: bapt.

LGTM beside small comment

tools/build/options/makeman.lua
349

probably the have a nor mal if then instead of if not then would be more straight forward

This revision is now accepted and ready to land.Apr 7 2023, 8:25 AM
kevans marked an inline comment as done.

Reverse sense of if

Fix an inconsistency with makeman: some showconfig invocations ignore dupes,
others take the latest value as the final value. Notably, implied-on options
end up taking the latest value. *waves hands*

This revision now requires review to proceed.Apr 18 2023, 3:52 AM

One source of duplicate options is share/mk/* and sys/conf/* having different defaults, see e.g. 7661a2abe45cfa144f69898f8498c3cfd88cceb8

We should warn when duplicates are found

Update past __REQUIRED_OPTIOMS and OPT_ multi handling

This is still a little hacky; I have some ideas on how to make it
more Lua-ish and less roughly transcribed shell script.

Add some linting for the default option run to note duplicate values that we
encounter

kevans edited the summary of this revision. (Show Details)

Discussed briefly with @kevans on IRC; the duplicate issues have been resolved with 269cbe092da38e1100df5008b664db89510c3604 I believe being the last one. I'm happy to go ahead with the switch to Lua and improve diagnostics over time.

tools/build/options/makeman.lua
37
38

As an aside if we get this to the point where it runs very quickly and we don't need to commit the generated src.conf.5 we'll need to sort out reproducible builds for .Dd. A problem for later on though.

155

OK for a direct translation from the sh version but I'd probably just leave AUTHORS out altogether.

Add a run_make() helper, switch from io.popen() to unistd.execp

We lose some convenience in popen using file handles and line-reading that
is already implemented as a result, but we cut out a number of trips through
/bin/sh as a result of doing this. I've pitched the thought of allowing
fbsd.exec() to capture stdout elsewhere and switching to that would perhaps be
a bit cleaner and a little more performant, but let's not rush that review.

This revision is now accepted and ready to land.May 28 2025, 11:06 PM
This revision was automatically updated to reflect the committed changes.