Page MenuHomeFreeBSD

lang/rust-bootstrap: Cross-compile Rust bootstraps for all archs
ClosedPublic

Authored by tobik on Jan 11 2020, 9:09 AM.

Details

Summary

Goals are:

  1. Putting the cost of building them on the FreeBSD project instead of volunteers
  2. Creating bootstraps for only supported releases instead of using whatever upstream builds them on (currently 10.3-RELEASE)
  3. Having the ability to easily patch them when it becomes necessary

Adding portmgr for approval of flavors.

Test Plan

Does lang/rust build with the new bootstraps?

amd64, i386: yes, tested by @tobik
aarch64: yes, tested by @garga (thanks!)
armv7: yes, tested by @mikael.urankar_gmail.com (thanks!)
armv6: untested
powerpc64 elfv1: yes. tested on ref12-ppc64 by @tobik, also tested by @pkubaj
powerpc64 elfv2: yes, tested by @pkubaj (thanks!)

Diff Detail

Repository
rP FreeBSD ports repository
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 28909
Build 26900: arc lint + arc unit

Event Timeline

tobik created this revision.Jan 11 2020, 9:09 AM
tobik updated this revision to Diff 66616.Jan 11 2020, 9:09 AM
  • Remove empty dir...
tobik updated this revision to Diff 66617.Jan 11 2020, 9:16 AM
  • Drop unused compiler-rt from distinfo
tobik updated this revision to Diff 66619.Jan 11 2020, 9:20 AM
  • Rebase
pkubaj accepted this revision.Jan 14 2020, 2:23 PM

Tested to build on both elfv1 and elfv2.

This revision is now accepted and ready to land.Jan 14 2020, 2:23 PM
mikael accepted this revision.Jan 14 2020, 2:35 PM

They finally fixed their cross-compile issue, I hope it won't break in the future.
It's ok on armv7, I'm afraid I won't have time to test on armv6 or aarch64 (but I see no reason it would break).

tobik edited the test plan for this revision. (Show Details)Jan 16 2020, 9:20 AM
garga added a subscriber: garga.Jan 16 2020, 1:32 PM
garga added a comment.Jan 17 2020, 3:31 PM

I tried to build lang/rust on native aarch64 builder with this patch applied, under a 11.3-STABLE jail and got:

running: /wrkdirs/usr/ports/lang/rust/work/rustc-1.40.0-src/build/aarch64-unknown-freebsd/stage0/bin/cargo build --manifest-path /wrkdirs/usr/ports/lang/rust/work/rustc-1.40.0-src/src/bootstrap/Cargo.toml --verbose --frozen
/wrkdirs/usr/ports/lang/rust/work/rustc-1.40.0-src/build/aarch64-unknown-freebsd/stage0/bin/cargo: Undefined symbol "lstat@FBSD_1.5"
Traceback (most recent call last):
  File "/wrkdirs/usr/ports/lang/rust/work/rustc-1.40.0-src/x.py", line 11, in <module>
    bootstrap.main()
  File "/wrkdirs/usr/ports/lang/rust/work/rustc-1.40.0-src/src/bootstrap/bootstrap.py", line 909, in main
    bootstrap(help_triggered)
  File "/wrkdirs/usr/ports/lang/rust/work/rustc-1.40.0-src/src/bootstrap/bootstrap.py", line 880, in bootstrap
    build.build_bootstrap()
  File "/wrkdirs/usr/ports/lang/rust/work/rustc-1.40.0-src/src/bootstrap/bootstrap.py", line 675, in build_bootstrap
    run(args, env=env, verbose=self.verbose)
  File "/wrkdirs/usr/ports/lang/rust/work/rustc-1.40.0-src/src/bootstrap/bootstrap.py", line 141, in run
    raise RuntimeError(err)
RuntimeError: failed to run: /wrkdirs/usr/ports/lang/rust/work/rustc-1.40.0-src/build/aarch64-unknown-freebsd/stage0/bin/cargo build --manifest-path /wrkdirs/usr/ports/lang/rust/work/rustc-1.40.0-src/src/bootstrap/Cargo.toml --verbose --frozen
*** Error code 1

Would it be possible to have a bootstrap compatible with FreeBSD 11?

Would it be possible to have a bootstrap compatible with FreeBSD 11?

You'll need to change _RUST_FBSD_VER_aarch64 to 11.3. Be aware that there are bugs in rtld on aarch64 11.3 (not sure if the bug is fixed in 11-stable), you'll need a ld-elf.so.1 from 12.1 or 12.stable (see bug https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=228892 for details)

tobik updated this revision to Diff 66963.Jan 18 2020, 9:34 AM
  • aarch64: Target 11.3-RELEASE (bootstrap already updated but give it an hour to appear on mirrors)
  • Add COMPAT_FREEBSD11 canary to lang/rust-bootstrap too
This revision now requires review to proceed.Jan 18 2020, 9:34 AM
tobik updated this revision to Diff 66964.Jan 18 2020, 9:57 AM
  • Rebase
tobik added a comment.Jan 23 2020, 9:16 AM

@garga So is it ok for you now?

@garga So is it ok for you now?

Unfortunately our ThunderX builder is out of service for maintenance and I won't be able to test it until next week. It is still broken for crossbuild, right?

@garga So is it ok for you now?

Unfortunately our ThunderX builder is out of service for maintenance and I won't be able to test it until next week.

Ok. Let me know how it goes.

It is still broken for crossbuild, right?

Yes, unfortunately the build will just hang with qemu-user-static at some point.

tobik updated this revision to Diff 67274.Jan 25 2020, 5:44 AM
  • Rebase

It fails on armv6, 11.3-RELEASE. I used to build the bootstrap for 11.3, it's 12.1 with this port. I'm not against it but we should restrict lang/rust accordingly. I'd prefer keeping support for 11.3 though.

running: /usr/ports/lang/rust/work/rustc-1.41.0-src/build/armv6-unknown-freebsd/stage0/bin/cargo build --manifest-path /usr/ports/lang/rust/work/rustc-1.41.0-src/src/bootstrap/Cargo.toml --verbose --frozen
/usr/ports/lang/rust/work/rustc-1.41.0-src/build/armv6-unknown-freebsd/stage0/bin/cargo: Undefined symbol "lstat@FBSD_1.5"
/usr/ports/lang/rust/work/rustc-1.41.0-src/build/armv6-unknown-freebsd/stage0/bin/cargo: ELF 32-bit LSB shared object, ARM, EABI5 version 1 (FreeBSD), dynamically linked, interpreter /libexec/ld-elf.so.1, for FreeBSD 12.1, FreeBSD-style, with debug_info, not stripped

Something is odd, it seems the armv6 sysroot is for armv7:

readelf -A /usr/ports/lang/rust-bootstrap/work-armv6/lib/libcrypt.so.5
Attribute Section: aeabi
File Attributes
  Tag_CPU_arch: ARM v7
  Tag_CPU_arch_profile: Application Profile
  Tag_ARM_ISA_use: Yes
  Tag_THUMB_ISA_use: 32-bit Thumb
  Tag_FP_arch: VFPv3
  Tag_Advanced_SIMD_arch: NEONv1
  Tag_CPU_unaligned_access: Not allowed
tobik added a comment.Jan 28 2020, 6:40 PM

Yeah, FreeBSD-11.3-RELEASE-arm-armv6.tar.xz and FreeBSD-12.1-RELEASE-arm-armv7.tar.xz were accidentally the same file... I'm rebuilding the armv6 bootstrap now.

tobik updated this revision to Diff 67430.Jan 28 2020, 7:48 PM
  • Actually build armv6 bootstrap for armv6

@garga So is it ok for you now?

Unfortunately our ThunderX builder is out of service for maintenance and I won't be able to test it until next week.

Ok. Let me know how it goes.

It built fine! Thanks!

tobik edited the test plan for this revision. (Show Details)Jan 29 2020, 8:25 AM
tobik updated this revision to Diff 67528.Jan 30 2020, 8:07 PM
  • Rebase and update to 1.41.0
tobik updated this revision to Diff 67529.Jan 30 2020, 8:09 PM
  • Lock lang/rust version to same version as lang/rust-bootstrap
tobik added a comment.Feb 15 2020, 3:12 PM

Can someone from portmgr approve the flavors? It would be nice if this could be committed before Rust 1.42.0 (2020-03-12).

mat added a comment.Feb 25 2020, 2:36 PM

So, this needs lang/rust to build, and, lang/rust needs it to build? Isn't there some sort of circular dependency?

lang/rust-bootstrap/Makefile
81

To be honest, I would rather you used a few .if than this hardly readable magic.

95–97

There are no patches, is this needed? Also, could the patch be named just like the flavor, it would be much more simpler.

tobik added a comment.Feb 25 2020, 2:58 PM
In D23133#523804, @mat wrote:

So, this needs lang/rust to build, and, lang/rust needs it to build? Isn't there some sort of circular dependency?

Yes, but there is no circular dependency in the ports tree sense. rust-bootstrap creates the bootstraps used for building the next stable release of lang/rust. I then upload the binaries somewhere before the next Rust release, so that lang/rust can fetch and use them. lang/rust will never depend directly on lang/rust-bootstrap.

lang/rust-bootstrap/Makefile
81

Fair enough. Would you also be fine with something like

_RUST_LLVM_TARGET= ${_RUST_LLVM_TARGET_${FLAVOR}}
_RUST_LLVM_TARGET_armv6= ARM
_RUST_LLVM_TARGET_armv7= ARM
...
95–97

There are patches. It reuses the ones from lang/rust.

tobik updated this revision to Diff 68806.Feb 25 2020, 3:23 PM
  • Make _RUST_LLVM_TARGET more readable
mat accepted this revision as: portmgr.Feb 25 2020, 4:24 PM

Accept the flavors, while I am here.

This revision was not accepted when it landed; it landed in state Needs Review.Feb 25 2020, 6:18 PM
Closed by commit rP527099: New port: lang/rust-bootstrap (authored by tobik). · Explain Why
This revision was automatically updated to reflect the committed changes.