Bug 260791 - lang/ruby30: Fails to link with CPUTYPE=haswell (2021Q4): ld: error: undefined symbol: _lzcnt_u32
Summary: lang/ruby30: Fails to link with CPUTYPE=haswell (2021Q4): ld: error: undefine...
Status: Closed FIXED
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: amd64 Any
: --- Affects Some People
Assignee: Jung-uk Kim
URL:
Keywords: needs-qa
Depends on:
Blocks:
 
Reported: 2021-12-29 10:35 UTC by Peter Much
Modified: 2022-04-23 17:41 UTC (History)
5 users (show)

See Also:
bugzilla: maintainer-feedback? (ruby)
koobs: merge-quarterly?


Attachments
fix amd64 detection (416 bytes, patch)
2022-04-03 15:54 UTC, Peter Much
no flags Details | Diff
lang/ruby30: suppor build with CPUTYPE=haswell and newer (661 bytes, patch)
2022-04-22 16:45 UTC, Vladimir Druzenko
vvd: maintainer-approval?
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Peter Much 2021-12-29 10:35:32 UTC
To reproduce:
-------------
set /etc/make.conf:
  CPUTYPE?=haswell

Error:
------
--- numeric.o ---
compiling numeric.c
In file included from numeric.c:35:
In file included from ./internal/numeric.h:13:
./internal/bits.h:240:26: warning: implicit declaration of function '_lzcnt_u32'
 is invalid in C99 [-Wimplicit-function-declaration]
    return (unsigned int)_lzcnt_u32(x);
                         ^
./internal/bits.h:269:26: warning: implicit declaration of function '_lzcnt_u64'
 is invalid in C99 [-Wimplicit-function-declaration]
    return (unsigned int)_lzcnt_u64(x);
                         ^
./internal/bits.h:455:22: warning: implicit declaration of function '_tzcnt_u32' is invalid in C99 [-Wimplicit-function-declaration]
    return (unsigned)_tzcnt_u32(x);
                     ^
./internal/bits.h:477:22: warning: implicit declaration of function '_tzcnt_u64' is invalid in C99 [-Wimplicit-function-declaration]
    return (unsigned)_tzcnt_u64(x);
                     ^
[...]
ld: error: undefined symbol: _lzcnt_u32
>>> referenced by bignum.c
>>>               bignum.o:(bary_divmod_normal)
>>> referenced by bignum.c
>>>               bignum.o:(rb_absint_size)
>>> referenced by bignum.c
>>>               bignum.o:(rb_absint_numwords)
>>> referenced by bignum.c
>>>               bignum.o:(rb_absint_numwords)
>>> referenced by bignum.c
>>>               bignum.o:(rb_int_parse_cstr)
>>> referenced by bignum.c
>>>               bignum.o:(rb_str2big_poweroftwo)
>>> referenced by bignum.c
>>>               bignum.o:(big2ulong)
>>> referenced by bignum.c
>>>               bignum.o:(big2ulong)
>>> referenced by bignum.c
>>>               bignum.o:(big2ull)
>>> referenced by bignum.c
>>>               bignum.o:(big2ull)
>>> referenced 18 more times

ld: error: undefined symbol: _tzcnt_u32
>>> referenced by compile.c
>>>               compile.o:(ibf_load_iseq_each)
>>> referenced by compile.c
>>>               compile.o:(ibf_load_iseq_each)
>>> referenced by compile.c
>>>               compile.o:(ibf_load_iseq_each)
>>> referenced by compile.c
>>>               compile.o:(ibf_load_iseq_each)
>>> referenced by compile.c
>>>               compile.o:(ibf_load_iseq_each)
>>> referenced by compile.c
>>>               compile.o:(ibf_load_iseq_each)
>>> referenced by compile.c
>>>               compile.o:(ibf_load_iseq_each)
>>> referenced by compile.c
>>>               compile.o:(ibf_load_iseq_each)
>>> referenced by compile.c
>>>               compile.o:(ibf_load_iseq_each)
>>> referenced by compile.c
>>>               compile.o:(ibf_load_iseq_each)
>>> referenced 66 more times

ld: error: undefined symbol: _lzcnt_u64
>>> referenced by numeric.c
>>>               numeric.o:(rb_int_bit_length)
>>> referenced by numeric.c
>>>               numeric.o:(rb_ulong_isqrt)
>>> referenced by numeric.c
>>>               numeric.o:(rb_int_s_isqrt)
>>> referenced by numeric.c
>>>               numeric.o:(builtin_inline_class_75)
>>> referenced by random.c
>>>               random.o:(random_ulong_limited)
>>> referenced by st.c
>>>               st.o:(rb_st_init_table_with_size)

ld: error: undefined symbol: _tzcnt_u64
>>> referenced by string.c
>>>               string.o:(rb_str_coderange_scan_restartable)
>>> referenced by string.c
>>>               string.o:(rb_str_coderange_scan_restartable)
>>> referenced by string.c
>>>               string.o:(rb_str_coderange_scan_restartable)
>>> referenced by string.c
>>>               string.o:(rb_external_str_new_with_enc)
>>> referenced by string.c
>>>               string.o:(rb_external_str_new_with_enc)
>>> referenced by string.c
>>>               string.o:(enc_strlen)
>>> referenced by string.c
>>>               string.o:(enc_strlen)
>>> referenced by string.c
>>>               string.o:(rb_enc_strlen_cr)
>>> referenced by string.c
>>>               string.o:(rb_enc_cr_str_copy_for_substr)
>>> referenced by string.c
>>>               string.o:(str_nth_len)
>>> referenced 5 more times
cc: error: linker command failed with exit code 1 (use -v to see invocation)
*** [miniruby] Error code 1

Analysis:
---------
internal/bits.h fails to include <x86intrin.h> because configure does not find it.
Configure does not find it because it does search for it only on certain 'x86_64' etc. machine, but we are 'amd64' machine:

configure.ac:
  AS_CASE("$target_cpu", [x64|x86_64|i[3-6]86*], [
    AC_CHECK_HEADERS(x86intrin.h)
  ])

---
configure:3327: checking target system type
configure:3340: result: amd64-portbld-freebsd12

I have no idea which party is wrong here, but adding 'amd64' into configure.ac solves the matter.
Comment 1 Kubilay Kocak freebsd_committer freebsd_triage 2022-01-17 00:13:46 UTC
@Reporter 

- Are the latest quarterly/head ports branches still affected?
- Are other lang/ruby* ports affected in the same way?
Comment 2 Peter Much 2022-01-17 14:13:27 UTC
Same flaw with 2022Q1 and head.
ruby26+27 did build okay.
Comment 3 Peter Much 2022-04-03 15:54:35 UTC
Created attachment 232918 [details]
fix amd64 detection

Problem does still exist with ruby31 in 2022Q2
Same fix.
Comment 4 Martin MATO 2022-04-21 21:30:27 UTC
As Ruby30  the default Now, it appears not compiling with CPUTYPE?=bdver3 defined in /etc/make.conf as well in one of my systems
undefining it and the port was compiling flawlessly.

The patch provided by Peter Much here solved the issue.

nota: strangely on one other rig (an old one) , the variable set to CPUTYPE?=core2 posed no problems...

Regards
Comment 5 Stefan Ehmann 2022-04-21 22:49:16 UTC
Same issue with CPUTYPE?=znver2 in ruby-3.0.4.

I'm using main, not quarterly branch.
Comment 6 Peter Much 2022-04-21 23:12:05 UTC
(In reply to Martin MATO from comment #4)
concerning core2: this is not surprising, as the respective functionality (bitwise operations) appears only in haswell and newer cpus.
Comment 7 Vladimir Druzenko freebsd_committer freebsd_triage 2022-04-22 16:45:10 UTC
Created attachment 233399 [details]
lang/ruby30: suppor build with CPUTYPE=haswell and newer

CPUTYPE?=sandybridge - and older build fine, haswell and newer - fails.

CFLAGS+=-mno-lzcnt doesn't help.

> adding 'amd64' into configure.ac solves 
Anybody report this to upstream?

https://bugs.ruby-lang.org/projects/ruby/wiki/HowToReport
https://bugs.ruby-lang.org/projects/ruby-master/issues
Comment 8 commit-hook freebsd_committer freebsd_triage 2022-04-23 03:12:54 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/ports/commit/?id=bd22c2827968b0ef6ef36dd853b31259c6ba3125

commit bd22c2827968b0ef6ef36dd853b31259c6ba3125
Author:     Jung-uk Kim <jkim@FreeBSD.org>
AuthorDate: 2022-04-23 03:04:46 +0000
Commit:     Jung-uk Kim <jkim@FreeBSD.org>
CommitDate: 2022-04-23 03:04:46 +0000

    lang/ruby3[0-2]: Fix build with certain CPUTYPE

    When CPUTYPE is set and the CPU supports lzcnt instruction, it fails to
    build because x86intrin.h is not included.  Fix the test to make it work
    on FreeBSD.

    PR:             260791
    Approved by:    ruby (maintainer timeout)

 lang/ruby30/files/patch-configure.ac | 17 +++++++++++++----
 lang/ruby31/files/patch-configure.ac | 15 ++++++++++++---
 lang/ruby32/files/patch-configure.ac | 15 ++++++++++++---
 3 files changed, 37 insertions(+), 10 deletions(-)
Comment 9 Jung-uk Kim freebsd_committer freebsd_triage 2022-04-23 03:17:43 UTC
I just went ahead and committed the fix for all lang/ruby3x ports.  As some people mentioned, this bug is more visible now because lang/ruby30 is default.
Comment 10 Yasuhiro Kimura freebsd_committer freebsd_triage 2022-04-23 16:11:23 UTC
I tried it on alderlake and the build error surely happens. But if I build Ruby 3.0.4 without using port (that is, tar xfpvJ ruby-3.0.4.txz; cd ruby-3.0.4; ./configure ; make) then it doesn't happen. Does this means that the problem happens only with FreeBSD Ruby ports? Does anybody succeeds in reproducing the build error with the latter way?
Comment 11 Stefan Ehmann 2022-04-23 17:06:44 UTC
(In reply to Yasuhiro Kimura from comment #10)

Can confirm that it only happens in ports build. I guess the difference is the detected arch in ports vs manual build:

Manual:
Configuration summary for ruby version 3.0.4

   * Installation prefix: /usr/local
   * exec prefix:         ${prefix}
   * arch:                x86_64-freebsd13

Ports:
Configuration summary for ruby version 3.0.4

   * Installation prefix: /usr/local
   * exec prefix:         ${prefix}
   * arch:                amd64-freebsd13
Comment 12 Yasuhiro Kimura freebsd_committer freebsd_triage 2022-04-23 17:41:40 UTC
(In reply to Stefan Ehmann from comment #11)

Thanks. Then it is very likely that this is not bug of Ruby itself but that of our Ruby ports. And if so upsream may not accept patch even if we send it to them.