Created attachment 226826 [details] v1 (use "git am") - add DTRACE option for user to decide lang/ruby26 assumes that DTrace support is guarded by exists() check, failing when base toolchain is removed, and instead external toolchain is provided by ports via /usr/local/share/toolchains/*.mk inclusion. - remove ${OPSYS} == "FreeBSD" check DTrace does not exist on DragonFly BSD, the only known consumer of FreeBSD ports. - Bump PORTREVISION per [1] poudriere testport (post-patch) Tier 1 arch logs: https://codeberg.org/ei/misc/commit/22ec334270277410067b24f9120022837cc1216d poudriere bulk diff (post-patch) Tier 1 arch logs: https://codeberg.org/ei/misc/commit/40a03285b80d6b49a49f8755ac97aefb493689b4 poudriere package diff (post-patch) Tier 1 arch logs [1]: https://codeberg.org/ei/misc/commit/8f6a59914e4acd5f57633935d07ae73985c0af08 Note, 11.4 has modified usr/local/lib/libruby* entries. This is not caused by this change, instead caused by any rebuild instead. Apparently, timestamp or something else gets baked in on 11.4, so 11.4 Ruby is not reproducible. I don't know any tool to diff binaries to tell what exactly is not reproducible. RUBY_PORTREVISION bump is not included in this patch, git commit --amend if committing. UPDATING entry is not needed, the default behaviour of port did not change, users who disable DTRACE globally will get notified via "options changed" of poudriere. lang/ruby27 change should follow shortly.
(In reply to Evgeniy Khramtsov from comment #0) > lang/ruby26 assumes that DTrace support is guarded by exists() check, failing when base toolchain is removed, Are you saying that /usr/sbin/dtrace doesn't exist if WITHOUT_TOOLCHAIN is set?
(In reply to Steve Wills from comment #1) WITHOUT_TOOLCHAIN is problematic (removes base headers). Instead, I use: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=257622#c2 dtrace does exist on such a system, but the port can't link DTRACE using non-base toolchain. Base toolchain links symbols that ports toolchain can't, e.g. https://github.com/DankBSD/ports/commit/fef8031 Some ports already use DTRACE option, e.g. DTRACE_DESC in Mk/bsd.options.desc.mk, so I think adding a DTRACE option to workaround this may be fine.
(In reply to Evgeniy Khramtsov from comment #2) > the port can't link Stderr is in the patch.
Maintainer timeout (1 month). Maybe add this option for ports tree consistency, e.g. do the same as databases/memcached, databases/postgresql14-server, lang/perl5.34, lang/erlang-runtime24, lang/ponyc, www/node, and many other ports that have optional DTRACE?
(In reply to Evgeniy Khramtsov from comment #2) > dtrace does exist on such a system, but the port can't link DTRACE using non-base toolchain. Why not? What is the error? Was it posted previously and I'm missing it? > Base toolchain links symbols that ports toolchain can't, e.g. https://github.com/DankBSD/ports/commit/fef8031 Sorry, I don't understand. The linked commit doesn't seem to explain much. I really don't understand this issue overall. It seems like it would really be helpful if you could be more specific about the issue(s) you seem to be encountering. You seem to be saying that you have set these options in src.conf: CROSS_TOOLCHAIN=llvm12 WITHOUT_CLANG=yes WITHOUT_CLANG_BOOTSTRAP=yes WITHOUT_LLD=yes WITHOUT_LLD_BOOTSTRAP=yes WITHOUT_LLDB=yes WITHOUT_LLVM_COV=yes and an additional patch to enable using external LLVM toolchain. What isn't clear is why you are doing that, or exactly what problem you are encountering due to this configuration. Perhaps if you could post the error message and/or full build log you get when trying to build ruby with this configuration it might help? I can say that ruby should be made reproducible if it isn't, in my opinion. I suggest using sysutils/py-diffoscope to investigate what is changing between builds; I've used it in the past and it's quite nice. I wouldn't suggest invoking maintainer timeout here until the issue is better understood. Maybe someone besides me understands it better. True the option to disable DTRACE is harmless, at the same time, we try not to add unnecessary options as each one adds a maintenance work in the long term. I'd apply the same logic to similarly filed DTRACE option patches.
(In reply to Steve Wills from comment #5) I did compare multiple builds of ruby with diffoscope and dtrace does seem to be a source of non-reproducibility in the package. So that seems like a good reason for a DTRACE option, at least until we can fix dtrace to use reproducible symbol names. But I don't understand why using the external compiler would cause the link issue, the symbols should be present regardless of which compiler builds them.
(In reply to Steve Wills from comment #5) > What is the error? Was it posted previously and I'm missing it? See v1 commit message: compiling dmyenc.c linking miniruby ld: error: undefined symbol: __dtraceenabled_ruby___array__create >>> referenced by array.c >>> array.o:(ary_new) >>> referenced by array.c >>> array.o:(rb_ary_new) >>> referenced by array.c >>> array.o:(rb_assoc_new) >>> referenced 27 more times > The linked commit doesn't seem to explain much. > It seems like it would really be helpful if you could be more specific about the issue(s) you seem to be encountering. > You seem to be saying that you have set these [...] options in src.conf > and an additional patch to enable using external LLVM toolchain. FreeBSD base system includes LLVM based toolchain, and if one follows CURRENT, then each update from source can take hours due to LLVM build. While there are different ways to solve this via meta mode or ccache, the most efficient way to lessen compile times is to not build the base toolchain, and use devel/llvm12 from ports instead. LLVM from ports can also be fetched using poudriere-devel pkg seeding via '-b' option, resulting in no compile time for LLVM. This allows me to build both kernel+world in less than 5 minutes using cheap (QTJ2) hardware. I did the same for my poudriere build jail and installed devel/llvm12 as toolchain. The ports LLVM != base LLVM; the base one can link DTrace symbols (in www/node, etc) while ports one can't. Maybe the ports LLVM can't link them due to diff between base LLVM, or the base LLVM includes some headers by default, or I don't have related toolchain binaries in $PATH symlinked to /usr/local/llvm12/*. This is not an issue to me; I don't use DTRACE options in ports and opted to turn them off instead. The only ports that I use that have non-optional DTRACE are lang/ruby2*. > Full build log you get when trying to build ruby with this configuration it might help? Sorry, but I can't expose my environment, as this is an infoleak that can help crafting exploits and puts me in risk. I prefer to have unique binaries. I already posted the required steps to reproduce this. > True the option to disable DTRACE is harmless, at the same time, we try not to add unnecessary options as each one adds a maintenance work in the long term. Currently DTrace support is an "automagic" option (which some folks dislike), but I think that the main rationale to include DTRACE as an option, is ports tree consistency. My use case happens to need this. > sysutils/py-diffoscope Thanks.
(In reply to Evgeniy Khramtsov from comment #7) Could you try running the scripts in this repo with your setup: https://github.com/swills/dtrace_thing and posting the results? You can leave out the particular md5 checksums if you want, but it would be helpful to know if they produce the same results on multiple runs or not. I wonder if this is a problem with the ports tool chain, this might help show that. I'm fine with adding the DTRACE option if we can better understand the issue and perhaps file a bug for the root cause, but simply turning it off as a workaround without understanding the issue seems like a bad idea, to me.
I've tried to reproduce this issue, but so far I've been unable to reproduce it.
(In reply to Steve Wills from comment #8) Sure. Although I changed $COMP because "cc" and "ld" don't exist here: -COMP="cc" +COMP="/usr/local/llvm12/bin/clang" and -COMP="c++" +COMP="/usr/local/llvm12/bin/clang++" 1 output: rm: app: No such file or directory rm: app.o: No such file or directory rm: probes.o: No such file or directory sh: ld: not found dtrace: failed to link script probes: failed to link probes.o: ld exited with status 127 clang: error: no such file or directory: 'probes.o' dtrace: failed to execute ./app: No such file or directory MD5 (app.o) = 4aa6f8c53e1786b9af667f38c03631de md5: probes.o: No such file or directory md5: app: No such file or directory 2 output: rm: app: No such file or directory rm: app.o: No such file or directory rm: probes.o: No such file or directory rm: probes.h: No such file or directory sh: ld: not found dtrace: failed to link script probes: failed to link probes.o: ld exited with status 127 clang++: error: no such file or directory: 'probes.o' dtrace: failed to execute ./app: No such file or directory MD5 (app.o) = e6c17b37918f8d953da79811b8131c0b md5: probes.o: No such file or directory md5: app: No such file or directory Seems that dtrace calls "ld" regardless of what is set as LD in /usr/share/mk/sys.mk. LD set via env(1) is also ignored. Maybe a base issue? Greg V of DankBSD could not experience issues with Ruby because of symlinks: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=257621#c1 In my setup I don't use symlinks.
(In reply to Evgeniy Khramtsov from comment #10) > Seems that dtrace calls "ld" regardless of what is set as LD in /usr/share/mk/sys.mk. LD set via env(1) is also ignored. Maybe a base issue? Thank you, yes, this is the cause of the issue and what I was trying to get to. Indeed, dtrace expects an "ld" command: https://cgit.freebsd.org/src/tree/cddl/contrib/opensolaris/lib/libdtrace/common/dt_open.c#n789 It would be nice if it checked LD environment variable, and we can file a bug requesting that, but expecting an "ld" command to exist somewhere seems like a pretty reasonable expectation to me. I'm not even sure where the right place to file a bug on dtrace would be. What is the upstream source? Or perhaps it should be filed against FreeBSD base since this was added here: https://cgit.freebsd.org/src/commit/cddl/contrib/opensolaris/lib/libdtrace/common/dt_open.c?id=fb24b944be9a0a5c0fe8a09b28321d1ff2c245fd Seems like there are a number of ways around the issue for now. You could patch this hard coded path in your setup for now, since you've already patched so many other things. Or perhaps create a patch to dtrace to have it check LD environment variable and fall back to "ld" and add that to a dtrace bug report. But, since we now understand the issue and disabling dtrace in the port avoids it and also helps with reproducibility, I'm OK with the patch to ruby to add the DTRACE option. Thanks again for the help understanding the issue. I'll try to file a bug against FreeBSD base and create a patch when I have some time.
Created attachment 227715 [details] patch to ruby port Could you test this patch and see if it allows ruby to build with dtrace enabled with your environment (non-system compiler, no system ld, etc.)?
(In reply to Steve Wills from comment #12) Sorry, never mind, that one seemed to work for ruby27 for me, but not for ruby26.
(In reply to Steve Wills from comment #13) Actually it does seem to work with ruby26, I was hitting a MAKE_JOBS issue locally when testing. If you could have a look, that'd be helpful.
(In reply to Steve Wills from comment #14) Adding "-x ldpath=/usr/local/llvm12/bin/ld" fixes examples: 1: dtrace: script 'script.d' matched 1 probe a=1, b=2 a=2, b=3 dtrace: pid 92114 has exited 1 2 1 2 3 1 MD5 (app.o) = 4aa6f8c53e1786b9af667f38c03631de MD5 (probes.o) = 9e27e17d7782614b07441578d49be643 MD5 (app) = 72cb8fb8f57b04d5c85405fc211c9e2a 2: rm: app: No such file or directory rm: app.o: No such file or directory rm: probes.o: No such file or directory rm: probes.h: No such file or directory dtrace: script 'script.d' matched 1 probe a=1, b=2 a=2, b=3 dtrace: pid 47097 has exited 1 2 1 2 3 1 MD5 (app.o) = e6c17b37918f8d953da79811b8131c0b MD5 (probes.o) = 2ee59d958960e03ec4772462606a0c4f MD5 (app) = 258f7505f4884f955ecc809d419f71e8 But it seems that Ruby needs something more? [...] dtrace: failed to compile script probes.d: Preprocessor not found [...] I took a look at build log: [...] checking whether dtrace USDT is available... yes(-x ldpath=ld -xnolibs) [...] LD = ld [...] Then I passed LD: - CONFIGURE_ENV= debugflags= + CONFIGURE_ENV= debugflags= LD=${LD} And the build log changed: [...] - checking whether dtrace USDT is available... yes(-x ldpath=ld -xnolibs) + checking whether dtrace USDT is available... yes(-x ldpath=/usr/local/llvm12/bin/ld -xnolibs) - checking whether dtrace needs post processing... no + checking whether dtrace needs post processing... rebuild [...] BASERUBY = echo executable host ruby is required. use --with-baseruby option.; false CC = /usr/local/llvm12/bin/clang - LD = ld + LD = /usr/local/llvm12/bin/ld LDSHARED = /usr/local/llvm12/bin/clang -shared CFLAGS = -O2 -pipe -O3 -mbranches-within-32B-boundaries -fstack-protector-all -fPIE -fPIC -isystem /usr/local/include -fno-strict-aliasing -fPIC XCFLAGS = -D_FORTIFY_SOURCE=2 -fstack-protector-strong -fno-strict-overflow -fvisibility=hidden -DRUBY_EXPORT -DCANONICALIZATION_FOR_MATHN CPPFLAGS = -I/usr/local/include -isystem /usr/local/include -I. -I.ext/include/amd64-freebsd14 -I./include -I. -I./enc/unicode/12.1.0 DLDFLAGS = -fstack-protector-all -pie -Wl,-z,noexecstack -Wl,-z,now -Wl,-z,relro -Wl,--compress-debug-sections=zlib -Wl,-E -Wl,-soname,libruby26.so.26 -fstack-protector-strong SOLIBS = -lelf -lunwind -lexecinfo -lprocstat -lz -lthr -lrt -lrt -ldl -lcrypt -lm -lpthread -L/usr/local/lib -L/usr/local/lib LANG = C LC_ALL = C LC_CTYPE = clang version 12.0.1 Target: x86_64-portbld-freebsd14.0 Thread model: posix InstalledDir: /usr/local/llvm12/bin translating probes probes.d dtrace: failed to compile script probes.d: Preprocessor not found *** Error code 1 ^ But still preprocessor (is it the $CPP one?) could not be found. Passing $CPP to CONFIGURE_ENV did not help. Did you try in a freshly created poudriere jail built using the patch and src.conf? The jail should not have "cc", etc by default. "delete-old" on existing jail does not delete "cc" and other binaries. Thanks for trying to fix this. My priority in fixing DTRACE with ports LLVM is low, but I will try any patches.
(In reply to Evgeniy Khramtsov from comment #15) Nice, I forgot to remove CFLAGS and other things by accident. Well, it doesn't matter now, here is the whole log (without make.conf): https://codeberg.org/ei/misc/commit/862e8732b5ffff
(In reply to Evgeniy Khramtsov from comment #16) Oh, when I tested I didn't remove cpp also. I'll try that and see if I can come up with an updated patch.
Created attachment 227718 [details] updated patch to ruby port Here's an updated patch that seems to fix the cpp issue. Not ideal, but works. Need someone who knows autotools better to really make it good enough to consider up-streaming.
Created attachment 227720 [details] dtrace patch It built. But I had to also pass LD via CONFIGURE_ENV. The working diff is attached. If I don't report back about ruby27, then assume it also built fine.
(In reply to Evgeniy Khramtsov from comment #19) I have switched to lang/ruby30 where this problem does not exist (no DTrace). My interest to test/rebase/update ruby2{6,7} patches in PRs may diminish.
This seems similar to PR 260831, which is for Ruby 2.6, 2.7 and 3.0 ?
Closing this bug as I don't need this anymore and someone is also willing to work on this. *** This bug has been marked as a duplicate of bug 260831 ***