I'm seeing a vaguely similar (crc32) error on arm64 (ut0crc32.h:135:10: fatal error: 'asm/hwcap.h' file not found)

[00:06:28] --- storage/innobase/CMakeFiles/innodb_zipdecompress.dir/all ---
[00:06:28] --- storage/innobase/CMakeFiles/innodb_zipdecompress.dir/buf/checksum.cc.o ---
[00:06:28] In file included from /wrkdirs/usr/ports/databases/mysql80-server/work/mysql-8.0.35/storage/innobase/buf/checksum.cc:36:
[00:06:28] In file included from /wrkdirs/usr/ports/databases/mysql80-server/work/mysql-8.0.35/storage/innobase/include/buf0buf.h:36:
[00:06:28] In file included from /wrkdirs/usr/ports/databases/mysql80-server/work/mysql-8.0.35/storage/innobase/include/buf0types.h:40:
[00:06:28] In file included from /wrkdirs/usr/ports/databases/mysql80-server/work/mysql-8.0.35/storage/innobase/include/ut0rnd.h:44:
[00:06:28] /wrkdirs/usr/ports/databases/mysql80-server/work/mysql-8.0.35/storage/innobase/include/ut0crc32.h:135:10: fatal error: 'asm/hwcap.h' file not found
[00:06:28] #include <asm/hwcap.h>
[00:06:28]          ^~~~~~~~~~~~~
[00:06:29] 1 error generated.
[00:06:29] *** [storage/innobase/CMakeFiles/innodb_zipdecompress.dir/buf/checksum.cc.o] Error code 1
[00:06:29] 
[00:06:29] make[3]: stopped in /wrkdirs/usr/ports/databases/mysql80-server/work/.build
[00:06:29] *** [all] Error code 6
[00:06:29] 
[00:06:29] make[1]: stopped in /wrkdirs/usr/ports/databases/mysql80-server/work/.build
[00:06:29] 1 error
[00:06:29] 
[00:06:29] make[1]: stopped in /wrkdirs/usr/ports/databases/mysql80-server/work/.build

Comment 2 Jochen Neumeister 2023-12-01 05:51:31 UTC

Can you please check it with this update? https://cgit.freebsd.org/ports/commit/?id=bbc7f225adae1c20fa9a4d3e28e544eb51730075

Also please look into my crc32 intrinsic issue that this bugreport is for. It is unrelated to the trivial header include error that the hijacking reply described, and I don't see it being addressed in that linked commit.

(In reply to Jochen Neumeister from comment #2)
Sorry, no unfortunately that doesn't fix it

(In reply to Viktor Štujber from comment #3)
Sorry I thought they might be the same issue. I'll make a new PR.

Comment 6 Jochen Neumeister 2023-12-02 06:59:25 UTC

(In reply to Viktor Štujber from comment #3)

on all current amd64 versions I can build the current MySQL. Likewise, pkg-fallout does not report any problems. Unfortunately, I cannot reproduce the problem on amd64.
also in my FreeBSD setup I currently have no problems with amd64

(In reply to Jochen Neumeister from comment #6)
As stated, this is freebsd 14.0 compiled for 'native', targeting an Intel D525MWV SoC cpu from Q4'2010, identifying itself as
CPU: Intel(R) Atom(TM) CPU D525   @ 1.80GHz (1800.14-MHz K8-class CPU)
Origin="GenuineIntel"  Id=0x106ca  Family=0x6  Model=0x1c  Stepping=10
Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
Features2=0x40e31d<SSE3,DTES64,MON,DS_CPL,TM2,SSSE3,CX16,xTPR,PDCM,MOVBE>

To directly reproduce on any modern pc, you'd probably have to cross-compile the toolchain, and then use that to cross-compile mysqld. Or not. I've never had to do that.

I have looked at mysqld /storage/innobase/ut/crc32.cc and found
#ifdef CRC32_x86_64
MY_ATTRIBUTE((target("sse4.2")))
uint32_t crc32_impl::update(uint32_t crc, unsigned char data) {
  return _mm_crc32_u8(crc, data);
}

These intrinsics are defined in /usr/lib/clang/16/include/crc32intrin.h
#define __DEFAULT_FN_ATTRS                                                     \
  __attribute__((__always_inline__, __nodebug__, __target__("crc32")))
static __inline__ unsigned int __DEFAULT_FN_ATTRS _mm_crc32_u8(unsigned int __C, unsigned char __D)

I have written a small test program
#include <crc32intrin.h>
int main()
{
    return _mm_crc32_u32(0, 0);
}

When I compile this, I get
test.cpp:4:2: error: always_inline function '_mm_crc32_u32' requires target feature 'crc32', but would be inlined into function 'main' that is compiled without support for 'crc32'

If I add __attribute__((target("sse4.2"))) or compile with -msse4.2, it builds, and when run it core dumps with 'Illegal instruction'.

I have dug into /storage/innobase/include/ut0crc32.h and found
#if defined(__GNUC__) && defined(__x86_64__) || defined(_WIN32)
#define CRC32_x86_64
#ifdef _WIN32
#define CRC32_x86_64_WIN
#else /* _WIN32 */
#define CRC32_x86_64_DEFAULT
#endif /* _WIN32 */

/* At this point we have classified the system statically into exactly one of
the possible cases:

CRC32_x86_64
    An environment in which we can use `cpuid` instruction to detect if it has
    support for crc32 and pclmul instructions, which (if available) can be used
    via _mm_crc32_u64 and _mm_clmulepi64_si128 respectively exposed by
    nmmintrin.h and wmmintrin.h.
    This is narrowed further into one of:

    CRC32_x86_64_WIN
        An environment which seems to be like Visual Studio, so we expect
        intrin.h header exposing `__cpuid`, which we can use instead of inline
        assembly, which is good as Visual Studio dialect of asm is different.
        Also, __attribute__(target(...)) probably doesn't work on it.
    CRC32_x86_64_DEFAULT
        An environment which seems to be like gcc or clang, and thus we can use
        inline assembly to get `cpuid`.
        Also, we can/have to use __attribute__(target(...)) on functions which
        use intrinsics, and may need to use __attribute__(flatten) at top level
        to ensure that the run-time selection of target-specific variant of the
        function happens just once at the top, not in every leaf, which would
        break inlining and optimizations.

So the mysql devs at least seem to be trying to ensure this builds on all platforms, including those that don't have hardware crc32 support. It's possible they gave up on that idea and now demand a 2016+ intel cpu, or they goofed and broke this backward compatibility, or their build system is somehow ifdefing around freebsd, or one of the freebsd ports patches is interfering. I have not yet had the spare time to consult the mysql devs about this. Hoping someone is staying on top of the current developments and can look into it.

It seems that the sse4.2 code has aways been there, but it's wrapped in ifdef macros and there's a CRC32_DEFAULT branch for pure software implementation. I don't know what changed between 8.0.32 and .35. Maybe it was something external, such as freebsd 13 clang not defining __GNUC__, causing a fall-through to CRC32_DEFAULT. Don't know, would have to invest a lot of time into bisecting and test-building this. I also saw that the freebsd port is applying patches specifically to the offending crc32 header and source file, which further complicates things.
Anyways, the only relevant commit on the mysql side seems to be https://github.com/mysql/mysql-server/commit/624f5847ef56c6a47e864c504d8bbc74335ba213

I got the same error.

CPU: AMD Athlon(tm) 64 X2 Dual Core Processor 5200+ (2712.70-MHz K8-class CPU)
  Origin="AuthenticAMD"  Id=0x60fb2  Family=0xf  Model=0x6b  Stepping=2
Features=0x178bfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,MMX,FXSR,SSE,SSE2,HTT>
  Features2=0x2001<SSE3,CX16>
  AMD Features=0xea500800<SYSCALL,NX,MMX+,FFXSR,RDTSCP,LM,3DNow!+,3DNow!>
  AMD Features2=0x11f<LAHF,CMP,SVM,ExtAPIC,CR8,Prefetch>
  SVM: (disabled in BIOS) NAsids=64
real memory  = 4294967296 (4096 MB)
avail memory = 4047663104 (3860 MB)

Prehistory.
I upgraded from 12.4 to 13.2 via freebsd-update, rebuilt databases/mysql57-server, but received another error (the bug report was already opened). I switched to mysql-8.0.35, got this error, read this bug report.
At this point, both mysql-servers are not compiled from ports.

I compiled it with the "CFLAGS= -DCRC32_DEFAULT" option in /etc/make.conf, but the database update from 5.7 to 8.0 failed due to errors.