I tried it first with llvm15, and then with llvm17 -- the error is the same: ... = note: ld: error: relocation R_X86_64_32 cannot be used against local symbol; recompile with -fPIC >>> defined in /cache/symbion/ports/lang/rust/work/_build/x86_64-unknown-freebsd/stage1-tools/x86_64-unknown-freebsd/release/deps/liblibssh2_sys-5419d5ae34e3db0f.rlib(knownhost.o) >>> referenced by knownhost.o:(libssh2_knownhost_init) in archive /cache/symbion/ports/lang/rust/work/_build/x86_64-unknown-freebsd/stage1-tools/x86_64-unknown-freebsd/release/deps/liblibssh2_sys-5419d5ae34e3db0f.rlib ld: error: relocation R_X86_64_32 cannot be used against local symbol; recompile with -fPIC >>> defined in /cache/symbion/ports/lang/rust/work/_build/x86_64-unknown-freebsd/stage1-tools/x86_64-unknown-freebsd/release/deps/liblibssh2_sys-5419d5ae34e3db0f.rlib(transport.o) >>> referenced by transport.o:(_libssh2_transport_send) in archive /cache/symbion/ports/lang/rust/work/_build/x86_64-unknown-freebsd/stage1-tools/x86_64-unknown-freebsd/release/deps/liblibssh2_sys-5419d5ae34e3db0f.rlib ld: error: relocation R_X86_64_32 cannot be used against local symbol; recompile with -fPIC >>> defined in /cache/symbion/ports/lang/rust/work/_build/x86_64-unknown-freebsd/stage1-tools/x86_64-unknown-freebsd/release/deps/liblibssh2_sys-5419d5ae34e3db0f.rlib(version.o) >>> referenced by version.o:(libssh2_version) in archive /cache/symbion/ports/lang/rust/work/_build/x86_64-unknown-freebsd/stage1-tools/x86_64-unknown-freebsd/release/deps/liblibssh2_sys-5419d5ae34e3db0f.rlib ... The machine is amd64, running stable/13-542497245d. llvm13 is also installed -- and cannot be deleted because of multiple dependencies. But the DEFAULT_VERSIONS contains llvm=17...
Is it still an issue?
I just reproduced this very same issue. Tried to build rust with PORT_LLVM on, and experienced the same relocation errors as in the original report. So I think this issue is still a problem, and I can only build rust without PORT_LLVM. My system: FreeBSD xxxx.xxx 13.2-STABLE FreeBSD 13.2-STABLE stable/13-n256726-7c25a53a2cb9 GENERIC amd64 Ports: git describe output: 13.1-eol-14216-g7e50d55c75b1 LLVM15 package info: Name : llvm15 Version : 15.0.7_7 Installed on : Sat Nov 25 23:10:49 2023 MST Origin : devel/llvm15 Architecture : FreeBSD:13:amd64 Prefix : /usr/local Categories : lang devel Licenses : REGEX and PD and MIT and LLVM2 and LLVM and BSD3CLAUSE Maintainer : brooks@FreeBSD.org WWW : https://llvm.org/ Comment : LLVM and Clang Options : BE_AMDGPU : on BE_FREEBSD : off BE_NATIVE : off BE_STANDARD : on BE_WASM : on CLANG : on COMPILER_RT : on DOCS : on EXTRAS : on FLANG : on GOLD : on LIT : on LLD : on LLDB : on MLIR : on OPENMP : on PYCLANG : on STATIC_LIBS : on Rust options are set to exclude WebAssembly (I think this used to be the default, and I never enabled it when WASM was set to be on by default). Rust builds fine without PORT_LLVM, but gets the same errors as the original issue report with PORT_LLVM on. I have nothing pertaining to LLVM or Rust in make.conf default versions variables.
(In reply to Mikael Urankar from comment #1) > Is it still an issue? My most recent attempt was with a 14.x system, when I attempted to build rust against the just-installed llvm16. Why, was this supposed to be fixed by something?
Maybe with rust 1.74.1
> Maybe with rust 1.74.1 Nope. Just tried -- same error in the same part (ssh2 something). The full build log is about 17Mb, so I don't want to attach it. Maybe, you try building it yourself?
I ran into this issue as well today when trying to update rust. Since the previous build of rust 1.73 on fresh 14.0 system succeeded, I tried removing libssh2 that had been installed as dependency for something else. After removing libssh2, rust with PORT_LLVM (version 17) builds again without the relocation errors.
(In reply to wenej27156 from comment #6) On my system, libssh2 was pulled in by cargo-c and libgit2, and cargo-c turns out to be a build dependency for a number of ports on my system. I'm reluctant to remove it just for rust's sake, since it'll just get reinstalled anyway the next time any of those other ports are updated. But it's good to know where the problem is coming from. There were other issues just recently with rust's build being borked by things like static libraries installed by assimp, among others (c.f. Bug 274723). Those issues were "fixed" by making the other package not install those libraries, but that's just working around rust's build system trying to pull in libraries it can't use just because it finds them present on the system. Here we seem to have the same problem.
lang/rust should probably switch to system libgit2/libssh for consistency with USES=cargo and to avoid ABI mismatch due to -I order[1]. Unfortunately, devel/libgit2 updates are often blocked by GitLab compatibility (e.g., bug 252098). [1] -I/usr/local/include -I. (or similar) picks up system API over bundled. Caused by https://wiki.freebsd.org/WarnerLosh/UsrLocal#Include_paths Usually fixed by -isystem via USES=localbase.
Created attachment 247014 [details] LIB_DEPEND on git2 and ssh2 The attached patch side-steps the miscompilation problem by explicitly depending on libgit2.so and libssh2.so (in addition to libcurl.so). The patch also prevents the bundled sources for the above (as well as for OpenSSL and llvm) from even being extracted from the tarball -- to prevent the possibility of compiling against the headers of one version of a library, then linking with another. It'd be nice to similarly eliminate the bundled code for xz-5.2, but currently the tools like rust-installer insist on /static/ linkage, and I'm not sufficiently proficient with the cargo-syntax to patch that. But this patch seems like a substantial improvement for the port already -- and may even allow to turn the PORT_LLVM option on /by default/.
(In reply to Mikhail Teterin from comment #9) I'm opposed to this change, I've spent countless hours in the past debugging problems due to the use of an old libgit2 in ports and don't want to go through that again.
(In reply to Mikael Urankar from comment #10) > I've spent countless hours The amount of effort is seldom a good argument... > problems due to the use of an old libgit2 in ports You can add a minimal version-requirement to a LIB_DEPENDency, will that work? The current "solution" -- using the bundled libgit2 and libssh2 -- violates the best-practices guidance^, and necessitates the "countless hours" of CPU-time as thousands of computers world-wide rebuild the LLVM bundled with rust -- in addition to LLVM from its own port (in addition to LLVM from src/). And, as this very ticket demonstrates, "countless hours" of HUMAN time is spent figuring out, why enabling PORT_LLVM (which should always be enabled anyway^) breaks things. ^ https://docs.freebsd.org/en/books/porters-handbook/special/#bundled-libs
Created attachment 251044 [details] LIB_DEPENDS on git2, ssh, nghttp2, and sqlite3 This is an updated patch with the 3rd-party code not even extract from the tarball, to avoid a possible conflict between headers and libraries. With this I'm able to build rust-1.78 with the port-provided LLVM on FreeBSD-14.1/stable (tested on both i386 and amd64). Frankly, I'm puzzled, why it is not considered a scandal, that rust port insists on rebuilding the bundled LLVM. Not only is such bundling against the guidelines, it also takes FOREVER!
Created attachment 251593 [details] Updated for 1.79.0 Why are these changes being ignored -- despite the active work on the port?
Created attachment 251614 [details] Updated for 1.79.0, excludes a bit more, when extracting
Bumped into this with rust-1.81.0. Nothing fancy in my make.conf and it was working for previous versions.
I 'only' have 16Gb of memory on my system, even with the build restricted to one core it OOM's after four hours, should I consider 16Gb insufficient for building Rust? This is looking crazy; hardly a great advert for Rust that it's build system is so bloated. Could you at least please remove the 'use system llvm' option if you do not intend to support it.
See https://reviews.freebsd.org/D32654#737459 FreeBSD unlike Linux doesn't default to PIE building in ports [1] which affects relocations in shared objects and causes mismatch because rustc implicitly enables PIE for parts of itself. D32654 landed as ports 098de5bc2195 which didn't include ${REINPLACE_CMD} 's|"-pie"|"-no-pie"|' from D32654 for related files. [1] https://cgit.freebsd.org/ports/tree/Mk/bsd.port.mk?id=0d89ee636380#n1019 PORT_LLVM=on works at the moment when pie.mk is active e.g. WITH_PIE defined in make.conf(5), so option description probably should note something alike "experimental and PIE-only" to avoid confusion by users. I'm not interested to troubleshoot non-PIE builds for an experimental option as: 1) I am not interested in non-PIE builds. 2) Low interest from rust@ for experimental areas may lead to reviews stalling endlessly and having to rebase continuously until a committer steps in. (Rust is a high effort port, and some users misleadingly expect **experimental** stuff to work for them then rant in PR (words like "scandal" don't help), may contribute to burnout of volunteers maintaining the port, so rust@ being hesitant to commit dragons is understandable at least.)
(In reply to Evgenii Khramtsov from comment #17) > words like "scandal" don't help To elaborate on that... Refreshing a FreeBSD desktop from source -- the BSD way -- involves the following steps: 1. Rebuild world and kernel, which includes rebuilding llvm/clang 2. Rebuild/upgrade ports, which almost always includes Thunderbird and Firefox. Rebuilding the Mozilla components requires rebuilding both llvm/clang and rust. And the latter requires rebuilding llvm YET AGAIN. Thus, a typical system upgrade requires recompiling LLVM _three times_ nowadays: 1. As part of the OS (src) 2. As part of LLVM-update itself. 3. As part of the Rust-update. There is nothing we, the ports-people, can do about base not installing LLVM -- though it is built as part of the "world", only the compiler tool-chain is _installed_, for whatever reasons. But we can -- and should! -- eliminate the need for the 3rd rebuild. By ensuring, Rust uses the existing LLVM (as well as other components, like ssh2). It is a good idea. And it is also the law: https://docs.freebsd.org/en/books/porters-handbook/special/#bundled-libs That it is not done -- despite available patches -- is not Ok. Whether it rises to the level of "scandal", is a matter of taste. > -no-pie I believe, the patch I proposed earlier (and then updated for the newer Rust version) side-steps this "pie" controversy entirely by making additional components use the existing ports instead of the bundled code... Have you tried it?
The handbook is not as "law" as you think it is. Specific exceptions do and will continue to exist. PORT_LLVM is not a default option and should not be default. The Rust project upstream themselves do not link to external LLVM by default; support for such there is maintained on a best-effort basis through llvm-wrapper. It's not a perfect situation but certainly much better than when they tracked their own fork of LLVM trunk prior to tracking stable releases starting LLVM 8.0. Best-effort means that, unless you are literally part of the rustc development process, you may not realise until you try to link rust 1.81 with LLVM 19 that the build bonks out, for example. Most importantly, note that Rust upstream do specify the minimum LLVM version required. For 1.81 and 1.82, this is LLVM 17. 1.83 will bump the minimum requirement to 18. I am also not interested in investigating non-PIE builds/environments for the time being. In my own copy of this port, I have it linking to security/libssh2 but not devel/libgit2 (not sure if the two(!) vendored libgit2-sys crates are used by anything these days).
> The handbook is not as "law" as you think it is A young polity, FreeBSD is still ruled by men, rather than laws -- and the men in charge will sneer at any attempt to change that as "lawyering"... But, whether it is a binding law, or just a collection of good ideas, Porter's Handbook is still something we all should be following. This particular principle -- the NOT using bundled third-party libraries -- is so clear-cut and non-controversial, projects like Gentoo and Fedora have adopted it too. > Specific exceptions do and will continue to exist. None of them applicable in this case... Your only counter-argument is that you're "not interested". That's plainly not valid... > PORT_LLVM is not a default option and should not be default Not only should using the existing llvm be the default, there is no reason for it to be an _option_ at all!.. > For 1.81 and 1.82, this is LLVM 17. 1.83 will bump the minimum requirement to 18. The port can specify this requirement too, if needed. The maximum version can be set too, if ever found to be necessary. Trivially done. > not sure if the two(!) vendored libgit2-sys crates are used by anything these days Let's add them to the --exclude list then, so they aren't even extracted by tar!
(In reply to Mikhail T. from comment #20) Good thing you're mentioning Gentoo, because it happens that I use Gentoo as well. So, Gentoo provides system-llvm USE flag for Rust which accepts the same as our PORT_LLVM. But: 1. It's off by default, 2. It's masked on stable, because: # Georgy Yakovlev <gyakovlev@gentoo.org> (2019-05-10) # needs llvm slots which are not stable #678908 # also prone to weird compilation failures # masking this for now, will readjust as situation changes So your argument is invalid. I have no idea about Fedora, but I wouldn't be surprised if it were the same.
> So, Gentoo provides system-llvm USE flag for Rust which accepts the same as our > PORT_LLVM. But: > 1. It's off by default, Yes, and FreeBSD also flipped into and out of using the llvm port throughout history, because of various problems popping up. I'd argue, it was not a good solution earlier -- those problems should've been addressed -- but, more importantly, it is not a solution now, when the problems don't even exist. But the reason I brought up Gentoo was their adoption of the "bundled code is bad" principle. That their developers violate it too, is neither here not there: https://bugs.gentoo.org/735154 > So your argument is invalid. Though it is not /necessary/ for an argument to have these to be invalid, it is certainly /sufficient/ for it to contain phrases like "I'm not interested" or "weird compilation issues". :-) To reiterate: 1. Rust supports building with pre-installed LLVM. 2. FreeBSD guidelines discourage use of bundled libraries. 3. Rust comes with a set of (non-)regression tests to identify build-failures. Conclusion: it is only FUD, that keeps us from doing the right thing. > I have no idea about Fedora I don't think, we should be taking a "vote", but Fedora's Rust RPM requires LLVM: http://www.rpmfind.net/linux/RPM/fedora/updates/testing/41/aarch64/Packages/r/rust-1.82.0-1.fc41.aarch64.html
Created attachment 259897 [details] Patch updated for rust-1.86.0 Bundled code explicitly excluded from extraction
friendly ping to inquire how's this going? fyi i've been using the bits and pieces found in this PR with great success for years now though a dozen rust versions, handfuls of major and minor LLVM versions (currently 22.1.3), many thousands of builds of many hundreds of applicable ports. (tbf i dont use it for -nightly tho; not that i've ran into any problems trying, simply not that devout a masochist to even try in the first place.) i also link this against system libssh2, libgit2, libnghttp2, libz, xz, openssl (3.6 here) and sqlite3 (or thats my intent, by the looks of my local diff; tbh i dont scrutinize the package when it just works.) (In reply to Mikael Urankar from comment #10) with respect to this very valid concern, i've encountered very little pain in this case; though perhaps planets do align during my lucky irregular update windows (not sarcastic). (In reply to Mikhail Teterin from comment #12) not a scandal; FPHB is chock full of "ideal philosophy" while our code tree also reflects a whole lot of "practical reality". rustaceans like to "move fast and break things" although i do wonder if rust isn't finally approaching a level of maturity where we can make most/all of the above libs follow FPHB § 6.3 [maybe even by default, just probly not in -nightly]. either way, i respect the rust@ team has more insight here than dozens of mes. admittedly tho :shrug: i'm filthy stinkin' rich and have RAM and CPU cores just fallin' outta my ears (praise Almighty) on the other hand i grew up on 64 KB and 2 MHz at 300 baud (uphill both ways in the snow and all that jazz lol) meaning i do want to promote efficiency over bloat, so empathy where due (In reply to Owen Carter from comment #16) rustaceans dont care; their mob of zealots are their advertisement. they top out at 24 years of age and their stepdad bought their computers. j/k i think rust is the bees knees and the community is...a fun bunch for sure
The option is working fine for me for a while. It also worked with DEFAULT_VERSIONS=llvm=21. Let's close this?
> Let's close this? Let's make the LLVM a default first :-) Also, I have these two "untracked" patches in the port's files/ directory, meant to force the use of existing lzma and nghttp2. But I haven't retested the need for them in a while... --- vendor/lzma-sys-0.1.20/build.rs 2025-03-31 17:37:24.000000000 -0400 +++ vendor/lzma-sys-0.1.20/build.rs 2025-04-26 13:50:57.912951000 -0400 @@ -6,4 +6,6 @@ fn main() { + println!("cargo:rustc-link-lib=lzma"); + return; let target = env::var("TARGET").unwrap(); --- vendor/libnghttp2-sys-0.1.11+1.64.0/build.rs +++ vendor/libnghttp2-sys-0.1.11+1.64.0/build.rs @@ -6,4 +6,5 @@ fn main() { + return; let target = env::var("TARGET").unwrap(); let out_dir = PathBuf::from(env::var_os("OUT_DIR").unwrap());