lang/rust: rework the do-build phase

[mikael]

Summary

Use x.py dist instead of doing the build / install dance (Inspired by OpenBSD)
It saves 1h on my aarch64 board (4cpus, 4GB of RAM):
before: build time: 05:30:22
after: build time: 04:28:30

Various cleanup / style fix.

Test Plan

The plist is the same between the two implementations
rust-nightly still builds

[tobik]

Do you know what the underlying issue with x.py build and x.py install is? It has never really been clear to me.

Reading back src/bootstrap/CHANGELOG.md, src/bootstrap/defaults/config.user.toml, and config.toml.example I think it is because build-stage and doc-stage default to 1 but install-stage defaults to 2 which causes Rust to rebuild some things during the install phase. But that is just conjecture. I wonder how long the build takes with the attached patch instead on your aarch64 board? Can you give it a try?

On my Azure VM the build on 13.0 amd64 goes down from 1:42 h to 1:29 h. With your patch the build time is 1:31 h, so about the same.

rust.diff1 KBDownload

[mikael]

In D28520#657426, @tobik wrote:

Do you know what the underlying issue with x.py build and x.py install is? It has never really been clear to me.

Honestly I don't know, it seems to only be an issue on low core machine (or at -j1). How many cpus do you have on your VM?

Reading back src/bootstrap/CHANGELOG.md, src/bootstrap/defaults/config.user.toml, and config.toml.example I think it is because build-stage and doc-stage default to 1 but install-stage defaults to 2 which causes Rust to rebuild some things during the install phase. But that is just conjecture. I wonder how long the build takes with the attached patch instead on your aarch64 board? Can you give it a try?

I'll try your patch tuesday.

FWIW, if you look at the ppc64 build log (only the lines after "===> Staging for rust-1.50.0"), it'll save more than an hour (I know it's not a very scientific benchmark but I see the same thing on aarch64):

grep -i 'Finished' rust-1.50.0.log
    Finished dev [unoptimized + debuginfo] target(s) in 2.19s
    Finished release [optimized] target(s) in 2.19s
To force LLVM to rebuild, remove the file `/wrkdirs/usr/ports/lang/rust/work/rustc-1.50.0-src/build/powerpc64-unknown-freebsd/llvm/llvm-finished-building`
    Finished release [optimized] target(s) in 3.10s
    Finished release [optimized] target(s) in 2.30s
    Finished release [optimized] target(s) in 3.02s
    Finished release [optimized] target(s) in 5m 22s
        finished in 62.155 seconds
    Finished release [optimized] target(s) in 5m 51s
        finished in 47.117 seconds
    Finished release [optimized] target(s) in 37m 42s
    Finished release [optimized] target(s) in 2m 30s
        finished in 31.250 seconds
    Finished release [optimized] target(s) in 17m 45s
    Finished release [optimized] target(s) in 46m 25s
        finished in 42.064 seconds
    Finished release [optimized] target(s) in 20m 29s
    Finished release [optimized] target(s) in 3.17s
        finished in 28.676 seconds
    Finished release [optimized] target(s) in 2.39s
        finished in 18.165 seconds
        finished in 33.393 seconds
        finished in 32.138 seconds
        finished in 24.164 seconds
    Finished release [optimized] target(s) in 13m 24s
        finished in 173.956 seconds

[tobik]

In D28520#657680, @mikael wrote:

In D28520#657426, @tobik wrote:

Do you know what the underlying issue with x.py build and x.py install is? It has never really been clear to me.

Honestly I don't know, it seems to only be an issue on low core machine (or at -j1). How many cpus do you have on your VM?

8 cores (D8s_v3)

Reading back src/bootstrap/CHANGELOG.md, src/bootstrap/defaults/config.user.toml, and config.toml.example I think it is because build-stage and doc-stage default to 1 but install-stage defaults to 2 which causes Rust to rebuild some things during the install phase. But that is just conjecture. I wonder how long the build takes with the attached patch instead on your aarch64 board? Can you give it a try?

I'll try your patch tuesday.

FWIW, if you look at the ppc64 build log (only the lines after "===> Staging for rust-1.50.0"), it'll save more than an hour (I know it's not a very scientific benchmark but I see the same thing on aarch64):

Well I see the same speed up too. The staging part of the build is basically reduced from 24 minutes to 3 minutes. Overall I have a build time that is ~85% of the original one (that is with either of the patches). The build times on your aarch64 board are also around ~85% of the original one. That's why I hope both patches do the same thing and we don't have to do something as complicated as OpenBSD.

[tobik]

@mikael, wondering why you abandoned? I am curious if adding build-stage=2 was already enough.

[mikael]

In D28520#675239, @tobik wrote:

@mikael, wondering why you abandoned? I am curious if adding build-stage=2 was already enough.

build-stage=2 made no improvement. I abandoned this because it adds too much complexity to the port.

[tobik]

In D28520#676669, @mikael wrote:

In D28520#675239, @tobik wrote:

@mikael, wondering why you abandoned? I am curious if adding build-stage=2 was already enough.

build-stage=2 made no improvement. I abandoned this because it adds too much complexity to the port.

Ok, fair enough.