Differential D32654
lang/rust: restore PORT_LLVM Authored by vishwin on Oct 26 2021, 2:19 AM. Tags None Referenced Files
Details
Allow Rust to build against ports LLVM.
llvm-config in cargo.toml implies linking to LLVM libraries, -Lnative is passed for linking to proceed. [1]: https://github.com/rust-lang/rust/issues/46437#issuecomment-426696032
Diff Detail
Event TimelineComment Actions It'd be absolutely fantastic if Rust would use LLVM from Ports, instead of compiling an own copy. Everyone using Poudriere to build stuff would benefit from it. This comment was removed by jbeich. Comment Actions
Fixed. llvm-config in config.toml resulted in rustc_codegen_ssa being passed LinkOutputKind::DynamicPicExe, which results in amended "-pie" for LD, failing to link with R_X86_64_32(S) relocations and the rest of non-PIE libraries: error: can't create dynamic relocation R_X86_64_32 against local symbol in readonly segment; recompile object files with -fPIC or pass '-Wl,-z,notext' to allow text relocations in the output Cosmetic changes (e.g. pass pre-link -no-pie in config.toml instead of extra patch) left. This now proved to work with more common non-PIE base/ports and might be committed, but I want to check if an extra patch can be avoided. I'll see this weekend. Comment Actions Last time I asked about this people mentioned this: Comment Actions
Passing '-no-pie' via config.toml affects the rest of rustc (e.g. stage0 build).
Don't build bundled LLD ("rust-lld", build enabled via config.toml as "lld=true") WASM target also hardcodes "rust-lld". While it can be overriden during runtime via $ rustc --target wasm32-unknown-unknown -O hello.rs Inspired by: Gentoo https://bugs.gentoo.org/715348 Comment Actions Note, I am not a Rust developer, but this is how I understand this:
"[rust] Add accessors for MCSubtargetInfo CPU and Feature tables": $ rg 'getFeatureTable' compiler/rustc_llvm/llvm-wrapper/PassWrapper.cpp PassWrapper.cpp, guarded by #ifdef LLVM_RUSTLLVM e.g.: extern "C" void LLVMRustGetTargetFeature(LLVMTargetMachineRef TM, size_t Index, const char** Feature, const char** Desc) {
[...]
const ArrayRef<SubtargetFeatureKV> FeatTable = MCInfo->getFeatureTable();
const SubtargetFeatureKV Feat = FeatTable[Index];
*Feature = Feat.Key;
*Desc = Feat.Desc;} $ rg 'LLVMRustGetTargetFeature' compiler/rustc_codegen_llvm/src/llvm_util.rs compiler/rustc_codegen_llvm/src/llvm/ffi.rs (FFI interface) llvm_util.rs, e.g.: llvm_target_features(tm: &llvm::TargetMachine) -> Vec<(&str, &str)> llvm::LLVMRustGetTargetFeature(tm, i, &mut feature, &mut desc);
if feature.is_null() || desc.is_null() {
bug!("LLVM returned a `null` target feature string");
}
let feature = CStr::from_ptr(feature).to_str().unwrap_or_else(|e| {
bug!("LLVM returned a non-utf8 feature string: {}", e);
});
let desc = CStr::from_ptr(desc).to_str().unwrap_or_else(|e| {
bug!("LLVM returned a non-utf8 feature string: {}", e);
});[...] $ rg 'llvm_target_features' compiler/rustc_codegen_llvm/src/llvm_util.rs llvm_util.rs: fn print_target_features(sess: &Session, tm: &llvm::TargetMachine) { let mut target_features = llvm_target_features(tm); [...] println!("Features supported by rustc for this target:");
for (feature, desc) in &rustc_target_features {
println!(" {1:0$} - {2}.", max_feature_len, feature, desc);
}[...] The sole user of getFeatureTable() of Rust's LLVM fork is rustc target
does not maintain related patches, and target_feature was switched from getFeatureTable [1]: https://github.com/rust-lang/rust/pull/49428 Also, -C target-cpu=skylake SIMD codegen here: $ rustc -C target-cpu=skylake --emit asm mat4.rs What I am concerned with: If LLVM doesn't maintain ABI compatibility per major release (I don't know), Comment Actions
Is being a committer a hard requirement? I am interested to use this as long as I use FreeBSD, and plan to fix possible PORT_LLVM was a non-default option for years before removal, and it wasn't always Besides, I marked this as "experimental" to avoid implying maintainance burden. Also, why aren't you interested? PORT_LLVM benefits are:
One would rather avoid building bundled LLVM when ports LLVM was already built. I get faster lang/rust build time with PORT_LLVM=on (~36 min. vs ~20 min. with on). Comment Actions Just put "unsupported" after the option description. Let's hope somebody supports it once it's in. Some of the interested committers in here can commit it. Comment Actions Unsupported might be understood as "not working". At least, that's what I would thought. Maybe "experimental"? Comment Actions I mean just use "community supported" then? Anyway use your own judgement. I'm not insisting on it. Comment Actions I need to warn that now I have no time for FOSS at all until somewhere in 2022Q2-2022Q3. I stopped Comment Actions
External LLVM is static by default in Rust's LLVM wrapper, e.g.: [...]
Comment Actions Commit message was lost and I have yet to learn to use Phabricator beyond Web frontend: From: Evgeniy Khramtsov <evgeniy@khramtsov.org> Rust bundles vendored LLVM for rarely essential changes, or features guarded behind LLVM_RUSTLLVM, that may not be required for a user. Upstream also provides a way to link against external LLVM, which can save build time when devel/llvmN was already built. Provide experimental tunable "LLVM_PORT" to allow building against LLVM from the FreeBSD ports collection. This tunable is unsupported and its use is discouraged. Inspired by: Gentoo Reviewed by: tobik Differential Revision: https://reviews.freebsd.org/D32654 --- Comment Actions Intermediate commit messages are not kept even when using arcinator (phab's command line interface), you have to explicitly put them in the comment/summary section Comment Actions
Obviously, not from www.rust-lang.org. Feel free to reword. Comment Actions This is Rust's documentation regarding updating and fixing bugs in their LLVM: https://rustc-dev-guide.rust-lang.org/backend/updating-llvm.html Comment Actions Enthusiasm to upstream lost. In FreeBSD, it seems that the separation between contributors and committers is too strong, as the latter
I also revoke this as I lost enthusiasm to fire poudriere for nothing, as there is no perspective of upstreaming. Without QA against baseline vanilla ports, I abandon this revision. Anyone interested is free to commandeer revision, I may reevaluate in future. Comment Actions rename back to PORT_LLVM, not least because USES=llvm claims the variable name. Per upstream, minimum external LLVM supported is 14; next major Rust version (1.73) bumps to 15. For this version (1.72), two upstream commits included to fix LLVM 17 API compatibility. On -CURRENT with LLVM 17, at least x11/alacritty and www/firefox (with LTO enabled) build and run as intended; the previous issues no longer exist due to tracking LLVM releases directly instead of snapshotting trunk.
I will support and maintain this bit. Comment Actions fwiw, this is still approved. Some optional suggestions below.
Comment Actions What's the real life approach with this change? Are we going to stick to using default version in ports tree of LLVM? If doable is that supported by upstream and if not who is going to take care of fallouts? If not, are we going to track upstream instead and is it worth the effort of unbundling LLVM as there like will be a handful of ports depending on the same (recent) version of LLVM? Comment Actions Saving cycles by using an external LLVM rather than compiling the entire bundled copy.
The bundled LLVM remains the default option, this is merely a use-at-own-risk convenience for those who want/have to build themselves.
The situation has changed drastically when this option was removed. Back then, which was during the LLVM 6.0/7.0 era, rust tracked LLVM trunk and was thus more sensitive to interactions between them, as evidenced with the small API breakage that eventually lead to the option removal. Starting with LLVM 8.0 and the advent of llvm-wrapper, rust switched to tracking releases with cherry-picking existing bug fix commits, much like we do with our base LLVM. Concurrently and consequently, external LLVM support has improved, with upstream tracking which LLVM version ranges work with each rust release.
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||