Page MenuHomeFreeBSD

lang/rust: Update to 1.52.0
ClosedPublic

Authored by tobik on May 4 2021, 8:37 AM.
Tags
None
Referenced Files
Unknown Object (File)
Tue, Mar 19, 9:48 PM
Unknown Object (File)
Tue, Mar 19, 2:31 PM
Unknown Object (File)
Thu, Mar 7, 12:47 AM
Unknown Object (File)
Thu, Mar 7, 12:47 AM
Unknown Object (File)
Thu, Mar 7, 12:47 AM
Unknown Object (File)
Thu, Mar 7, 12:47 AM
Unknown Object (File)
Thu, Mar 7, 12:47 AM
Unknown Object (File)
Thu, Mar 7, 12:47 AM

Details

Reviewers
None
Group Reviewers
rust
portmgr
Commits
R11:d8837418ea74: lang/rust: Update to 1.52.1
Summary

Scheduled to be released on 2021-04-06.

https://github.com/rust-lang/rust/blob/stable/RELEASES.md#version-1520-2021-05-06
https://internals.rust-lang.org/t/1-52-0-prerelease-testing/14628

Other changes:

  • Added an update script for lang/rust similar to lang/rust-nightly
  • Changed how USES=cargo generates DISTFILES and MASTER_SITES. See the associated commit message for more details. It's unrelated to the update but I've batched it together because of the exp-run
  • Use binutils strip on ppc64 ELFv1; base strip complains about unsupported e_type

I've also pushed this here https://github.com/t6/freebsd-ports/tree/tobik/rust-1.52.0

Test Plan

ref11-i386 ok
ref12-amd64 ok
ref12-ppc64 ok-ish (strip complains about unsupported e_type?)
ref13-ppc64 ok
ref13-aarch64 ok
ref13-amd64 ok

I'll leave rust-1.52.0 installed in /scratch/tmp/tobik/local/usr/local on the reference machines when the build was ok in case you need to check something.

armv6 ok
armv7 ok
powerpc64le ok

Exp-run: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=255608

Diff Detail

Repository
rP FreeBSD ports repository
Lint
No Lint Coverage
Unit
No Test Coverage
Build Status
Buildable 39066
Build 35955: arc lint + arc unit

Event Timeline

tobik held this revision as a draft.
  • Mk/Uses/cargo.mk: Refactor MASTER_SITES and DISTFILES construction
  • Mk/Uses/cargo.mk: Simplify crate matches
  • Use newer strip on ELFv1
tobik edited the test plan for this revision. (Show Details)
tobik published this revision for review.May 4 2021, 11:46 AM

Note: I have the bootstrap side working for 32 bit powerpc, been holding off on submitting it though because I haven't successfully gotten a full lang/rust build out of it yet (problems with linking.) I did upstream the obligatory openssl-src-rs crate changes needed to bootstrap though at https://github.com/alexcrichton/openssl-src-rs/commit/9cc3301e6168237d2b466cb92d5535e0a6252c51 .

I stuck it at https://people.freebsd.org/~bdragon/rust_bootstrap_ppc.patch if you want to do a 1.51 bootstrap build. Otherwise, when I have it fully working, I'll sneak in another X -> X bootstrap like we did for ppc64le initially.

Note: I have the bootstrap side working for 32 bit powerpc, been holding off on submitting it though because I haven't successfully gotten a full lang/rust build out of it yet (problems with linking.)

That's nice. What problems specifically?

I stuck it at https://people.freebsd.org/~bdragon/rust_bootstrap_ppc.patch if you want to do a 1.51 bootstrap build.

I've applied it. Your patch has the prebuilt bootstrap in distinfo but I could not find it in LOCAL/bdragon. Can you upload them? I can copy them to LOCAL/tobik then.

Even if it isn't working we can at least hook it up. I've marked powerpc as broken in lang/rust for now.

I did upstream the obligatory openssl-src-rs crate changes needed to bootstrap though at https://github.com/alexcrichton/openssl-src-rs/commit/9cc3301e6168237d2b466cb92d5535e0a6252c51 .

Can you upstream the LLVM, libc, and powerpc-unknown-freebsd target bits too?

tobik edited the summary of this revision. (Show Details)
  • Add 32-bit powerpc patches from bdragon

Note: I have the bootstrap side working for 32 bit powerpc, been holding off on submitting it though because I haven't successfully gotten a full lang/rust build out of it yet (problems with linking.)

That's nice. What problems specifically?

Out of range PLT relocs.

http://drop.rtk0.net/20210504.txt

jhibbits just clued me in that I wasn't enabling the longcall feature, so I'm going to retry with that.

I stuck it at https://people.freebsd.org/~bdragon/rust_bootstrap_ppc.patch if you want to do a 1.51 bootstrap build.

I've applied it. Your patch has the prebuilt bootstrap in distinfo but I could not find it in LOCAL/bdragon. Can you upload them? I can copy them to LOCAL/tobik then.

Yeah, I was going to upload them as soon as I got a lang/rust build to finish without bailing.

I am going to do a rebuild today though, jhibbits gave me some insight as to what's going wrong with the build currently, so I'm gonna do a rebootstrap with some slight targeting adjustments. (i.e. force -mlongcall.)

Even if it isn't working we can at least hook it up. I've marked powerpc as broken in lang/rust for now.

I did upstream the obligatory openssl-src-rs crate changes needed to bootstrap though at https://github.com/alexcrichton/openssl-src-rs/commit/9cc3301e6168237d2b466cb92d5535e0a6252c51 .

Can you upstream the LLVM, libc, and powerpc-unknown-freebsd target bits too?

Yeah, once I have things nailed down.

Note: I have the bootstrap side working for 32 bit powerpc, been holding off on submitting it though because I haven't successfully gotten a full lang/rust build out of it yet (problems with linking.)

That's nice. What problems specifically?

Out of range PLT relocs.

http://drop.rtk0.net/20210504.txt

It's looking like this problem was due to the rust build invoking the system compiler without the full FreeBSD version passed as the target. This meant that the system clang was defaulting to bss-plt, which wasn't able to set up branches due to the size of the binary.

I had encountered this same problem with rust-bootstrap and had fixed it there. I'm rebuilding the bootstrap with more minimal targeting changes to see if it works there without having to do stuff like patch the code model. If it works, I will tidy it up and submit that as the final 1.51.0 bootstrap.

Getting really close to having this fully up and running.

Most of the problems I've hit would have been avoided if we had per FreeBSD version rust target names. :-/

Finalized boostraps for powerpc uploaded to my public_distfiles on freefall.

I'm still working on some fixes for the lang/rust build (plus I need to move my patches over to 1.52.0 anyway) but I have gotten an eventually successful native build of patched 1.51.0 out of this bootstrap, so I'm calling it good for now.

SHA256 (rust/2021-02-11/rustc-1.51.0-powerpc-unknown-freebsd.tar.xz) = a95f8bae7685cf512d096f75846181bb90e6f35dc0bc29d68f4574520e7488f6
SIZE (rust/2021-02-11/rustc-1.51.0-powerpc-unknown-freebsd.tar.xz) = 47820404
SHA256 (rust/2021-02-11/rust-std-1.51.0-powerpc-unknown-freebsd.tar.xz) = bc7fffac703c5bf3c90325110f17c507e39f2ae795ef2894ba58a9ad6bb87004
SIZE (rust/2021-02-11/rust-std-1.51.0-powerpc-unknown-freebsd.tar.xz) = 17509168
SHA256 (rust/2021-02-11/cargo-1.51.0-powerpc-unknown-freebsd.tar.xz) = 96b5935082775f6ad882f6b8b27e57abc2df32efbe1e5746f998226ef5f2b20e
SIZE (rust/2021-02-11/cargo-1.51.0-powerpc-unknown-freebsd.tar.xz) = 4947120
  • Update distinfo with new powerpc bootstraps
  • Add LibreSSL 3.3.x patch from PR 255652

On powerpc64, things look indeed ok, apart from one thing (a regression back from 1.51.0).
Firefox-esr and Thunderbird both hang during build (happens also on powerpc64le)
Attaching full build log:

I wouldn't call it a showstopper, since the main Firefox port builds and works, but it would be nice to get it working.

Nevermind, things seem to work on 1.52.0, powerpc64le also is ok.

Final powerpc changes:

https://people.freebsd.org/~bdragon/rust-1.52-ppc-final-fixes.patch

I got a successful build with this for powerpc.

I've been thinking on this, and I think we should push for upstream to default to FreeBSD 13 ABIs on powerpc* platforms (ELFv2 / ppc32 secure-plt) so we can isolate the version specific patching in ports to only be needed for ELFv1. At least until they grow proper support for handling osversion.

  • Apply final powerpc fixes from bdragon
  • Use the right base for arc diff :-/

Hi,

I apologize if this is just noise but adding @${ECHO_CMD} 'codegen-units=1' >> ${WRKSRC}/config.toml seems help a lot on my machines when it comes to memory usage during compilation, is this something we can consider adding by default?

Here's a patch to the current version in tree (1.51.0) for more context:
https://projects.pyret.net/files/public/freebsd/rust-memory-codegen-units.patch

Best regards,
Daniel

A few things I've noted that other distros does that we might want to adopt? This is just a few differences I noticed while comparing our port to other distros, it's by no mean any attempt to hijack this patch.

Alpine,Arch,Gentoo,OpenSuse,Void uses LLVM from their package tree instead of bundled, perhaps that's worth looking into?

https://git.alpinelinux.org/aports/tree/community/rust/APKBUILD#n124
https://git.alpinelinux.org/aports/tree/community/rust/APKBUILD#n147
https://git.alpinelinux.org/aports/tree/community/rust/APKBUILD#n151
https://git.archlinux.org/svntogit/packages.git/tree/trunk/PKGBUILD?h=packages/rust#n56
https://build.opensuse.org/package/view_file/openSUSE:Factory/rust/rust.spec?expand=1 (L463)

Possibly related? https://github.com/void-linux/void-packages/blob/master/srcpkgs/rust/patches/00016-do-not-copy-libunwind.patch

Codegen-units defaults to 1 (Arch provides a bit more context)

https://git.alpinelinux.org/aports/tree/community/rust/APKBUILD#n157
https://git.alpinelinux.org/aports/tree/community/rust/APKBUILD#n158
https://git.archlinux.org/svntogit/packages.git/tree/trunk/PKGBUILD?h=packages/rust#n72
https://gitweb.gentoo.org/repo/gentoo.git/tree/dev-lang/rust/rust-1.52.0.ebuild#n317
https://github.com/void-linux/void-packages/blob/master/srcpkgs/rust/template#L243
https://github.com/void-linux/void-packages/blob/master/srcpkgs/rust/template#L244
https://build.opensuse.org/package/view_file/openSUSE:Factory/rust/rust.spec?expand=1

Disable parallel compiling, I guess this is to ease memory pressure
https://git.alpinelinux.org/aports/tree/community/rust/APKBUILD#n159
https://github.com/void-linux/void-packages/blob/master/srcpkgs/rust/template#L251

https://git.alpinelinux.org/aports/tree/community/rust/link-musl-dynamically.patch (adopt to our library?)
https://git.alpinelinux.org/aports/tree/community/rust/musl-fix-linux_musl_base.patch (same as above)
https://github.com/void-linux/void-packages/blob/master/srcpkgs/rust/patches/0009-Link-stage2-tools-dynamically-to-libstd.patch
https://gitweb.gentoo.org/repo/gentoo.git/tree/dev-lang/rust/rust-1.51.0-r2.ebuild#n323 (more static linking?)

Unbundle libssh2?
https://build.opensuse.org/package/view_file/openSUSE:Factory/rust/rust.spec?expand=1 (L507)

Thanks for keeping this up to date!

This comment was removed by pkubaj.
This revision was not accepted when it landed; it landed in state Needs Review.May 11 2021, 5:33 PM
This revision was automatically updated to reflect the committed changes.

@bdragon lang/rust-bootstrap@powerpc failed to build. I tried to fix it in 0ade1fd7a6cd7eef17d19cf118bf1ee15ec1c653 but I'm wondering how you managed to build it? Are we missing a patch or something else?

http://package19.nyi.freebsd.org/data/122amd64-default-build-as-user/ab31b591e5a6/logs/powerpc-rust-bootstrap-1.52.1_1.log