Page MenuHomeFreeBSD

lang/rust-bootstrap: Cross-compile Rust bootstraps for all archs
ClosedPublic

Authored by tobik on Jan 11 2020, 9:09 AM.

Details

Summary

Goals are:

  1. Putting the cost of building them on the FreeBSD project instead of volunteers
  2. Creating bootstraps for only supported releases instead of using whatever upstream builds them on (currently 10.3-RELEASE)
  3. Having the ability to easily patch them when it becomes necessary

Adding portmgr for approval of flavors.

Test Plan

Does lang/rust build with the new bootstraps?

amd64, i386: yes, tested by @tobik
aarch64: yes, tested by @garga (thanks!)
armv7: yes, tested by @mikael.urankar_gmail.com (thanks!)
armv6: untested
powerpc64 elfv1: yes. tested on ref12-ppc64 by @tobik, also tested by @pkubaj
powerpc64 elfv2: yes, tested by @pkubaj (thanks!)

Diff Detail

Repository
rP FreeBSD ports repository
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 28607
Build 26641: arc lint + arc unit

Event Timeline

  • Drop unused compiler-rt from distinfo

Tested to build on both elfv1 and elfv2.

This revision is now accepted and ready to land.Jan 14 2020, 2:23 PM

They finally fixed their cross-compile issue, I hope it won't break in the future.
It's ok on armv7, I'm afraid I won't have time to test on armv6 or aarch64 (but I see no reason it would break).

I tried to build lang/rust on native aarch64 builder with this patch applied, under a 11.3-STABLE jail and got:

running: /wrkdirs/usr/ports/lang/rust/work/rustc-1.40.0-src/build/aarch64-unknown-freebsd/stage0/bin/cargo build --manifest-path /wrkdirs/usr/ports/lang/rust/work/rustc-1.40.0-src/src/bootstrap/Cargo.toml --verbose --frozen
/wrkdirs/usr/ports/lang/rust/work/rustc-1.40.0-src/build/aarch64-unknown-freebsd/stage0/bin/cargo: Undefined symbol "lstat@FBSD_1.5"
Traceback (most recent call last):
  File "/wrkdirs/usr/ports/lang/rust/work/rustc-1.40.0-src/x.py", line 11, in <module>
    bootstrap.main()
  File "/wrkdirs/usr/ports/lang/rust/work/rustc-1.40.0-src/src/bootstrap/bootstrap.py", line 909, in main
    bootstrap(help_triggered)
  File "/wrkdirs/usr/ports/lang/rust/work/rustc-1.40.0-src/src/bootstrap/bootstrap.py", line 880, in bootstrap
    build.build_bootstrap()
  File "/wrkdirs/usr/ports/lang/rust/work/rustc-1.40.0-src/src/bootstrap/bootstrap.py", line 675, in build_bootstrap
    run(args, env=env, verbose=self.verbose)
  File "/wrkdirs/usr/ports/lang/rust/work/rustc-1.40.0-src/src/bootstrap/bootstrap.py", line 141, in run
    raise RuntimeError(err)
RuntimeError: failed to run: /wrkdirs/usr/ports/lang/rust/work/rustc-1.40.0-src/build/aarch64-unknown-freebsd/stage0/bin/cargo build --manifest-path /wrkdirs/usr/ports/lang/rust/work/rustc-1.40.0-src/src/bootstrap/Cargo.toml --verbose --frozen
*** Error code 1

Would it be possible to have a bootstrap compatible with FreeBSD 11?

Would it be possible to have a bootstrap compatible with FreeBSD 11?

You'll need to change _RUST_FBSD_VER_aarch64 to 11.3. Be aware that there are bugs in rtld on aarch64 11.3 (not sure if the bug is fixed in 11-stable), you'll need a ld-elf.so.1 from 12.1 or 12.stable (see bug https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=228892 for details)

  • aarch64: Target 11.3-RELEASE (bootstrap already updated but give it an hour to appear on mirrors)
  • Add COMPAT_FREEBSD11 canary to lang/rust-bootstrap too
This revision now requires review to proceed.Jan 18 2020, 9:34 AM

@garga So is it ok for you now?

Unfortunately our ThunderX builder is out of service for maintenance and I won't be able to test it until next week. It is still broken for crossbuild, right?

@garga So is it ok for you now?

Unfortunately our ThunderX builder is out of service for maintenance and I won't be able to test it until next week.

Ok. Let me know how it goes.

It is still broken for crossbuild, right?

Yes, unfortunately the build will just hang with qemu-user-static at some point.

It fails on armv6, 11.3-RELEASE. I used to build the bootstrap for 11.3, it's 12.1 with this port. I'm not against it but we should restrict lang/rust accordingly. I'd prefer keeping support for 11.3 though.

running: /usr/ports/lang/rust/work/rustc-1.41.0-src/build/armv6-unknown-freebsd/stage0/bin/cargo build --manifest-path /usr/ports/lang/rust/work/rustc-1.41.0-src/src/bootstrap/Cargo.toml --verbose --frozen
/usr/ports/lang/rust/work/rustc-1.41.0-src/build/armv6-unknown-freebsd/stage0/bin/cargo: Undefined symbol "lstat@FBSD_1.5"
/usr/ports/lang/rust/work/rustc-1.41.0-src/build/armv6-unknown-freebsd/stage0/bin/cargo: ELF 32-bit LSB shared object, ARM, EABI5 version 1 (FreeBSD), dynamically linked, interpreter /libexec/ld-elf.so.1, for FreeBSD 12.1, FreeBSD-style, with debug_info, not stripped

Something is odd, it seems the armv6 sysroot is for armv7:

readelf -A /usr/ports/lang/rust-bootstrap/work-armv6/lib/libcrypt.so.5
Attribute Section: aeabi
File Attributes
  Tag_CPU_arch: ARM v7
  Tag_CPU_arch_profile: Application Profile
  Tag_ARM_ISA_use: Yes
  Tag_THUMB_ISA_use: 32-bit Thumb
  Tag_FP_arch: VFPv3
  Tag_Advanced_SIMD_arch: NEONv1
  Tag_CPU_unaligned_access: Not allowed

Yeah, FreeBSD-11.3-RELEASE-arm-armv6.tar.xz and FreeBSD-12.1-RELEASE-arm-armv7.tar.xz were accidentally the same file... I'm rebuilding the armv6 bootstrap now.

  • Actually build armv6 bootstrap for armv6

@garga So is it ok for you now?

Unfortunately our ThunderX builder is out of service for maintenance and I won't be able to test it until next week.

Ok. Let me know how it goes.

It built fine! Thanks!

  • Rebase and update to 1.41.0
  • Lock lang/rust version to same version as lang/rust-bootstrap

Can someone from portmgr approve the flavors? It would be nice if this could be committed before Rust 1.42.0 (2020-03-12).

So, this needs lang/rust to build, and, lang/rust needs it to build? Isn't there some sort of circular dependency?

lang/rust-bootstrap/Makefile
80

To be honest, I would rather you used a few .if than this hardly readable magic.

94–96

There are no patches, is this needed? Also, could the patch be named just like the flavor, it would be much more simpler.

In D23133#523804, @mat wrote:

So, this needs lang/rust to build, and, lang/rust needs it to build? Isn't there some sort of circular dependency?

Yes, but there is no circular dependency in the ports tree sense. rust-bootstrap creates the bootstraps used for building the next stable release of lang/rust. I then upload the binaries somewhere before the next Rust release, so that lang/rust can fetch and use them. lang/rust will never depend directly on lang/rust-bootstrap.

lang/rust-bootstrap/Makefile
80

Fair enough. Would you also be fine with something like

_RUST_LLVM_TARGET= ${_RUST_LLVM_TARGET_${FLAVOR}}
_RUST_LLVM_TARGET_armv6= ARM
_RUST_LLVM_TARGET_armv7= ARM
...
94–96

There are patches. It reuses the ones from lang/rust.

  • Make _RUST_LLVM_TARGET more readable

Accept the flavors, while I am here.

This revision was not accepted when it landed; it landed in state Needs Review.Feb 25 2020, 6:18 PM
Closed by commit rP527099: New port: lang/rust-bootstrap (authored by tobik). · Explain Why
This revision was automatically updated to reflect the committed changes.