lang/rust-bootstrap: Cross-compile Rust bootstraps for all archs

[tobik]

Summary

Goals are:

1. Putting the cost of building them on the FreeBSD project instead of volunteers
2. Creating bootstraps for only supported releases instead of using whatever upstream builds them on (currently 10.3-RELEASE)
3. Having the ability to easily patch them when it becomes necessary

Adding portmgr for approval of flavors.

Test Plan

Does lang/rust build with the new bootstraps?

amd64, i386: yes, tested by @tobik
aarch64: yes, tested by @garga (thanks!)
armv7: yes, tested by @mikael.urankar_gmail.com (thanks!)
armv6: untested
powerpc64 elfv1: yes. tested on ref12-ppc64 by @tobik, also tested by @pkubaj
powerpc64 elfv2: yes, tested by @pkubaj (thanks!)

[tobik]

• Remove empty dir...

[tobik]

• Drop unused compiler-rt from distinfo

[tobik]

• Rebase

[pkubaj]

Tested to build on both elfv1 and elfv2.

[mikael]

They finally fixed their cross-compile issue, I hope it won't break in the future.
It's ok on armv7, I'm afraid I won't have time to test on armv6 or aarch64 (but I see no reason it would break).

[garga]

I tried to build lang/rust on native aarch64 builder with this patch applied, under a 11.3-STABLE jail and got:

running: /wrkdirs/usr/ports/lang/rust/work/rustc-1.40.0-src/build/aarch64-unknown-freebsd/stage0/bin/cargo build --manifest-path /wrkdirs/usr/ports/lang/rust/work/rustc-1.40.0-src/src/bootstrap/Cargo.toml --verbose --frozen
/wrkdirs/usr/ports/lang/rust/work/rustc-1.40.0-src/build/aarch64-unknown-freebsd/stage0/bin/cargo: Undefined symbol "lstat@FBSD_1.5"
Traceback (most recent call last):
  File "/wrkdirs/usr/ports/lang/rust/work/rustc-1.40.0-src/x.py", line 11, in <module>
    bootstrap.main()
  File "/wrkdirs/usr/ports/lang/rust/work/rustc-1.40.0-src/src/bootstrap/bootstrap.py", line 909, in main
    bootstrap(help_triggered)
  File "/wrkdirs/usr/ports/lang/rust/work/rustc-1.40.0-src/src/bootstrap/bootstrap.py", line 880, in bootstrap
    build.build_bootstrap()
  File "/wrkdirs/usr/ports/lang/rust/work/rustc-1.40.0-src/src/bootstrap/bootstrap.py", line 675, in build_bootstrap
    run(args, env=env, verbose=self.verbose)
  File "/wrkdirs/usr/ports/lang/rust/work/rustc-1.40.0-src/src/bootstrap/bootstrap.py", line 141, in run
    raise RuntimeError(err)
RuntimeError: failed to run: /wrkdirs/usr/ports/lang/rust/work/rustc-1.40.0-src/build/aarch64-unknown-freebsd/stage0/bin/cargo build --manifest-path /wrkdirs/usr/ports/lang/rust/work/rustc-1.40.0-src/src/bootstrap/Cargo.toml --verbose --frozen
*** Error code 1

Would it be possible to have a bootstrap compatible with FreeBSD 11?

[mikael]

In D23133#509306, @garga wrote:

Would it be possible to have a bootstrap compatible with FreeBSD 11?

You'll need to change _RUST_FBSD_VER_aarch64 to 11.3. Be aware that there are bugs in rtld on aarch64 11.3 (not sure if the bug is fixed in 11-stable), you'll need a ld-elf.so.1 from 12.1 or 12.stable (see bug https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=228892 for details)

[tobik]

• aarch64: Target 11.3-RELEASE (bootstrap already updated but give it an hour to appear on mirrors)
• Add COMPAT_FREEBSD11 canary to lang/rust-bootstrap too

[tobik]

• Rebase

[tobik]

@garga So is it ok for you now?

[garga]

In D23133#511441, @tobik wrote:

@garga So is it ok for you now?

Unfortunately our ThunderX builder is out of service for maintenance and I won't be able to test it until next week. It is still broken for crossbuild, right?

[tobik]

In D23133#511524, @garga wrote:

In D23133#511441, @tobik wrote:

@garga So is it ok for you now?

Unfortunately our ThunderX builder is out of service for maintenance and I won't be able to test it until next week.

Ok. Let me know how it goes.

It is still broken for crossbuild, right?

Yes, unfortunately the build will just hang with qemu-user-static at some point.

[tobik]

• Rebase

[tobik]

...

[mikael]

It fails on armv6, 11.3-RELEASE. I used to build the bootstrap for 11.3, it's 12.1 with this port. I'm not against it but we should restrict lang/rust accordingly. I'd prefer keeping support for 11.3 though.

running: /usr/ports/lang/rust/work/rustc-1.41.0-src/build/armv6-unknown-freebsd/stage0/bin/cargo build --manifest-path /usr/ports/lang/rust/work/rustc-1.41.0-src/src/bootstrap/Cargo.toml --verbose --frozen
/usr/ports/lang/rust/work/rustc-1.41.0-src/build/armv6-unknown-freebsd/stage0/bin/cargo: Undefined symbol "lstat@FBSD_1.5"
/usr/ports/lang/rust/work/rustc-1.41.0-src/build/armv6-unknown-freebsd/stage0/bin/cargo: ELF 32-bit LSB shared object, ARM, EABI5 version 1 (FreeBSD), dynamically linked, interpreter /libexec/ld-elf.so.1, for FreeBSD 12.1, FreeBSD-style, with debug_info, not stripped

[mikael]

Something is odd, it seems the armv6 sysroot is for armv7:

readelf -A /usr/ports/lang/rust-bootstrap/work-armv6/lib/libcrypt.so.5
Attribute Section: aeabi
File Attributes
  Tag_CPU_arch: ARM v7
  Tag_CPU_arch_profile: Application Profile
  Tag_ARM_ISA_use: Yes
  Tag_THUMB_ISA_use: 32-bit Thumb
  Tag_FP_arch: VFPv3
  Tag_Advanced_SIMD_arch: NEONv1
  Tag_CPU_unaligned_access: Not allowed

[tobik]

Yeah, FreeBSD-11.3-RELEASE-arm-armv6.tar.xz and FreeBSD-12.1-RELEASE-arm-armv7.tar.xz were accidentally the same file... I'm rebuilding the armv6 bootstrap now.

[tobik]

• Actually build armv6 bootstrap for armv6

[garga]

In D23133#511525, @tobik wrote:

In D23133#511524, @garga wrote:

In D23133#511441, @tobik wrote:

@garga So is it ok for you now?

Unfortunately our ThunderX builder is out of service for maintenance and I won't be able to test it until next week.

Ok. Let me know how it goes.

It built fine! Thanks!

[tobik]

• Rebase and update to 1.41.0

[tobik]

• Lock lang/rust version to same version as lang/rust-bootstrap

[tobik]

Can someone from portmgr approve the flavors? It would be nice if this could be committed before Rust 1.42.0 (2020-03-12).

[tobik]

Ping?

[mat]

So, this needs lang/rust to build, and, lang/rust needs it to build? Isn't there some sort of circular dependency?

lang/rust-bootstrap/Makefile

81

To be honest, I would rather you used a few .if than this hardly readable magic.

95–97

There are no patches, is this needed? Also, could the patch be named just like the flavor, it would be much more simpler.

[tobik]

In D23133#523804, @mat wrote:

So, this needs lang/rust to build, and, lang/rust needs it to build? Isn't there some sort of circular dependency?

Yes, but there is no circular dependency in the ports tree sense. rust-bootstrap creates the bootstraps used for building the next stable release of lang/rust. I then upload the binaries somewhere before the next Rust release, so that lang/rust can fetch and use them. lang/rust will never depend directly on lang/rust-bootstrap.

lang/rust-bootstrap/Makefile

81

Fair enough. Would you also be fine with something like

_RUST_LLVM_TARGET= ${_RUST_LLVM_TARGET_${FLAVOR}}
_RUST_LLVM_TARGET_armv6= ARM
_RUST_LLVM_TARGET_armv7= ARM
...

95–97

There are patches. It reuses the ones from lang/rust.

[tobik]

• Make _RUST_LLVM_TARGET more readable

[mat]

Accept the flavors, while I am here.