Page MenuHomeFreeBSD

framework: add support for using sccache during port builds
Needs ReviewPublic

Authored by tcberner on Jul 20 2021, 4:07 PM.
Tags
None
Referenced Files
Unknown Object (File)
Tue, Apr 16, 6:53 AM
Unknown Object (File)
Tue, Apr 16, 6:53 AM
Unknown Object (File)
Tue, Apr 16, 6:49 AM
Unknown Object (File)
Fri, Apr 12, 5:07 PM
Unknown Object (File)
Feb 10 2024, 6:51 PM
Unknown Object (File)
Feb 10 2024, 6:51 PM
Unknown Object (File)
Feb 10 2024, 6:51 PM
Unknown Object (File)
Feb 10 2024, 6:51 PM
Subscribers

Details

Reviewers
None
Group Reviewers
portmgr
rust
desktop
Summary

A big issue with lang/rust and its users is that building it takes
about three times longer than people have patience for.

Simlilar to ccache there is sccache which can be used for rust.
As this is a rust program, this is unfortunately a chicken and
egg problem, as opposed to ccache, we cannot depend on the sscache
port.

The newly added bsd.sscache.mk supports the following three variables:

  • SSCACHE_SIZE how much space do you want to sacrifice for time
  • SSCACHE_DIR what space do you want to sacrifice
  • SCCACHE_BIN the sccache binary you provide

Default values:

  • SCCACHE_SIZE defaults to 16G
  • SCCACHE_BIN defaults to SCCACHE_DIR/sccache if the latter is set

This makes it quite easy to inject sccache into poudriere, by abusing
the current ccache infrastructure (obviously this can be improved).
For this to work properly, the ccache dirs in poudriere have to be
split by architecture/version (as we put the sccache binary there in
this example):

Assuming the ccache used by the 14-CURRENT amd64 jail called 140amd64 is /zcache/ccache/140-64

i.e. you have

	CCACHE_DIR=/zcache/ccache/140-64

in the /usr/local/etc/poudriere.d/140amd64-poudriere.conf

Create a place for the sccache:

	> mkdir -p /zcache/ccache/140-64/sccache

Copy the pre-compiled sccache binary:

	> cp /usr/local/bin/sccache /zcache/ccache/140-64/sccache

Configure poudriere

	> echo SCCACHE_DIR=/root/.ccache/sccache >> /usr/local/etc/poudriere.d/140amd64-make.conf

This change makes it possible to compile for example lang/rust and
graphics/librsvg2-rust in poudriere quite fast:

	> poudriere bulk -j140amd64 -pmain -tC lang/rust graphics/librsvg2-rust

i.e.

	[...]
	[00:19:04] [01] [00:18:56] Finished lang/rust | rust-1.53.0: Success
	[00:20:30] [01] [00:01:22] Finished graphics/librsvg2-rust | librsvg2-rust-2.50.3_4: Success
	[...]

while using around 1G of disk space.

Note: I have no clue of rust and its build tools, so I might have missed
something and broken another :)

As an example x11/wezterm here gives a speep improvement of around 40%:

  • with sccache: around 4.5 minutes
  • without sccache: around 7 minutes

Diff Detail

Repository
rP FreeBSD ports repository
Lint
No Lint Coverage
Unit
No Test Coverage
Build Status
Buildable 40566
Build 37455: arc lint + arc unit

Event Timeline

tcberner retitled this revision from */*: add support for using sccache during port builds to framework: add support for using sccache during port builds.

Update commit message

  • fix cache size environment variable name
  • add post-build target to printout statistics of the cache
  • add SCCACHE_MAX_FRAME_LENGTH=104857600 to the environment [1]
  • remote the statistics print out
  • add sccache-start and -stop targets added to the CONFIGURE_ and STAGE_SEQ to ensure the sccache server gets shut down when the build stops.

[1] https://wiki.gentoo.org/wiki/Sccache#Initial_setup

tobik added inline comments.
Mk/bsd.port.mk
5335

Wondering if the sccache server needs to be started/stopped for the test phase too. Many USES=cargo ports come with tests and compiling them could probably benefit from sccache too.

Mk/bsd.sccache.mk
50

If the messages stay, let's drop the ====, use ${ECHO_MSG} , and start them with a capital letter.

lang/rust/Makefile
165

Is setting rustc-wrapper in config.toml necessary when we already set RUSTC_WRAPPER in the environment?

227

This is a nop block it seems. Did you mean SCCACHE_ENABLED?

228

FILES_DIR is not defined.

232

Same.

lang/rust/files/extra-patch-sccache
8–10

Those variables are already exported via MAKE_ENV in bsd.sccache.mk. Why do we need this?

I've added ports-mgmt/sccache-overlay which comes with a ports overlay with a sccache binary for the host that can also work inside of non-native jails. Everything just works (YMMV) once you enable the overlay and the normal ccache support in Poudriere.

Additionally c4816e4763b8d395b4a2cb670a8aaaf33bb8def1 will probably help across lang/rust updates.

I've added ports-mgmt/sccache-overlay which comes with a ports overlay with a sccache binary for the host that can also work inside of non-native jails. Everything just works (YMMV) once you enable the overlay and the normal ccache support in Poudriere.

Additionally c4816e4763b8d395b4a2cb670a8aaaf33bb8def1 will probably help across lang/rust updates.

Cool, thank you very much for this :)