Bug 261660

Summary: databases/mysql80-server: Fails to build on aarch64: innobase/ut/crc32.cc:114:10: fatal error: 'asm/hwcap.h' file not found
Product: Ports & Packages Reporter: tech-lists
Component: Individual Port(s)Assignee: Mikael Urankar <mikael>
Status: Closed FIXED    
Severity: Affects Many People CC: darius, mikael, tech-lists
Priority: --- Keywords: needs-qa
Version: LatestFlags: bugzilla: maintainer-feedback? (joneum)
koobs: merge-quarterly?
Hardware: arm64   
OS: Any   
See Also: https://github.com/xmrig/xmrig/issues/2330
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=246982
Attachments:
Description Flags
full poudriere log
none
Patch for ARM64 CRC code
none
Updated CRC32 patch
none
Smaller diff for CRC32/Crypto fixes
none
Patch port makefile to enable crc & crypto instructions none

Description tech-lists 2022-02-01 18:37:06 UTC
Created attachment 231500 [details]
full poudriere log

Hi,

context: 
main-n252544-7406ec4ea99 arm64 
poudriere-devel-3.3.99.20211130

PORTSTREE METHOD    TIMESTAMP           PATH
default   git+https 2022-02-01 11:02:16 /usr/local/poudriere/ports/default

poudriere testport for databases/mysql80-server errors here:

[00:12:41] cd /wrkdirs/usr/ports/databases/mysql80-server/work/.build/storage/myisam && /ccache/libexec/ccache/c++ -DHAVE_CONFIG_H -DLZ4_DISABLE_DEPRECATE_WARNINGS -DRAPIDJSON_NO_SIZETYPEDEFINE -DRAPIDJSON_SCHEMA_USE_INTERNALREGEX=0 -DRAPIDJSON_SCHEMA_USE_STDREGEX=1 -D_USE_MATH_DEFINES -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -I/wrkdirs/usr/ports/databases/mysql80-server/work/.build -I/wrkdirs/usr/ports/databases/mysql80-server/work/.build/include -I/wrkdirs/usr/ports/databases/mysql80-server/work/mysql-8.0.28 -I/wrkdirs/usr/ports/databases/mysql80-server/work/mysql-8.0.28/include -isystem /wrkdirs/usr/ports/databases/mysql80-server/work/mysql-8.0.28/extra/rapidjson/include -std=c++17 -fno-omit-frame-pointer -ftls-model=initial-exec -O2 -pipe  -fPIC -DNDEBUG -malign-double -fstack-protector-strong -isystem /usr/local/include -fno-strict-aliasing   -isystem /usr/local/include -std=c++17 -moutline-atomics -Wall -Wextra -Wformat-security -Wvla -Wundef -Wmissing-format-attribute -Woverloaded-virtual -Wcast-qual -Wno-null-conversion -Wno-unused-private-field -Wconditional-uninitialized -Wdeprecated -Wextra-semi -Wheader-hygiene -Wnon-virtual-dtor -Wundefined-reinterpret-cast -Winconsistent-missing-destructor-override -Winconsistent-missing-override -Wshadow-field -ffunction-sections -fdata-sections -O2 -pipe  -fPIC -DNDEBUG -malign-double -fstack-protector-strong -isystem /usr/local/include -fno-strict-aliasing   -isystem /usr/local/include -std=c++17 -fPIC -MD -MT storage/myisam/CMakeFiles/myisam_library.dir/mi_check.cc.o -MF CMakeFiles/myisam_library.dir/mi_check.cc.o.d -o CMakeFiles/myisam_library.dir/mi_check.cc.o -c /wrkdirs/usr/ports/databases/mysql80-server/work/mysql-8.0.28/storage/myisam/mi_check.cc
[00:12:42] --- storage/innobase/CMakeFiles/innodb_zipdecompress.dir/all ---
[00:12:42] --- storage/innobase/CMakeFiles/innodb_zipdecompress.dir/ut/crc32.cc.o ---
[00:12:42] /wrkdirs/usr/ports/databases/mysql80-server/work/mysql-8.0.28/storage/innobase/ut/crc32.cc:114:10: fatal error: 'asm/hwcap.h' file not found
[00:12:42] #include <asm/hwcap.h>
[00:12:42]          ^~~~~~~~~~~~~
[00:12:42] 1 error generated.
[00:12:42] *** [storage/innobase/CMakeFiles/innodb_zipdecompress.dir/ut/crc32.cc.o] Error code 1
Comment 1 tech-lists 2022-02-03 11:51:00 UTC
fails to build on arm64.aarch64
Comment 2 tech-lists 2022-02-07 14:57:13 UTC
asm/hwcap.h seems to be a linux thing. it's not on a freebsd-only installation.

It's in things which have been ported from linux like multimedia/vlc :

% locate hwcap.h
/usr/ports/multimedia/v4l_compat/work/linux-5.17-rc1/arch/arm/include/asm/hwcap.h
/usr/ports/multimedia/v4l_compat/work/linux-5.17-rc1/arch/arm/include/uapi/asm/hwcap.h
/usr/ports/multimedia/v4l_compat/work/linux-5.17-rc1/arch/arm64/include/asm/hwcap.h
/usr/ports/multimedia/v4l_compat/work/linux-5.17-rc1/arch/arm64/include/uapi/asm/hwcap.h
/usr/ports/multimedia/v4l_compat/work/linux-5.17-rc1/arch/mips/include/uapi/asm/hwcap.h
/usr/ports/multimedia/v4l_compat/work/linux-5.17-rc1/arch/riscv/include/asm/hwcap.h
/usr/ports/multimedia/v4l_compat/work/linux-5.17-rc1/arch/riscv/include/uapi/asm/hwcap.h

It's present on an ubuntu-server VM:

/usr/src/linux-headers-5.4.0-97/arch/arm/include/asm/hwcap.h
/usr/src/linux-headers-5.4.0-97/arch/arm/include/uapi/asm/hwcap.h
/usr/src/linux-headers-5.4.0-97/arch/arm64/include/asm/hwcap.h
/usr/src/linux-headers-5.4.0-97/arch/arm64/include/uapi/asm/hwcap.h
/usr/src/linux-headers-5.4.0-97/arch/mips/include/uapi/asm/hwcap.h
/usr/src/linux-headers-5.4.0-97/arch/riscv/include/asm/hwcap.h
/usr/src/linux-headers-5.4.0-97/arch/riscv/include/uapi/asm/hwcap.h
/usr/src/linux-headers-5.4.0-97/arch/unicore32/include/asm/hwcap.h
Comment 3 Daniel O'Connor 2022-02-12 23:33:24 UTC
Created attachment 231777 [details]
Patch for ARM64 CRC code
Comment 4 Daniel O'Connor 2022-02-12 23:34:30 UTC
The attached patch plus adding "-march=armv8.1-a+crc+crypto" to CFLAGS makes it compile for me.

I can't really test it though, I ran out of space on my SD card building it :(

Also not really sure how the CFLAGS change should be done..
Comment 5 tech-lists 2022-02-13 02:32:16 UTC
(In reply to Daniel O'Connor from comment #4)

Thank you Daniel, I will try your patch and let you know how it gets on

many thanks,
Comment 6 Daniel O'Connor 2022-02-13 12:21:53 UTC
I found some warnings like..

'++crc+crypto' is not a recognized feature for this target (ignoring feature)
'++crc+crypto' is not a recognized feature for this target (ignoring feature)
'++crc' is not a recognized feature for this target (ignoring feature)
'++crc' is not a recognized feature for this target (ignoring feature)
'++crypto' is not a recognized feature for this target (ignoring feature)
'++crypto' is not a recognized feature for this target (ignoring feature)

Which should be fixed with the latest version of the patch - I am not sure if they are a problem or not.

Still not sure how to apply the -march though.
Comment 7 Daniel O'Connor 2022-02-13 12:22:24 UTC
Created attachment 231790 [details]
Updated CRC32 patch
Comment 8 Mikael Urankar freebsd_committer freebsd_triage 2022-02-13 16:19:04 UTC
(In reply to Daniel O'Connor from comment #7)
Does it build fine with this patch ?
Comment 9 tech-lists 2022-02-15 01:30:58 UTC
(In reply to Mikael Urankar from comment #8)

unfortunately it doesn't work for me. I slightly changed the patch to look 
for 8.0.28 as that's the version now. 

I have CFLAGS+= -march=armv8.1-a+crc+crypto in the make.conf for this poudriere instance. Let me know if you need the entire build log.

The patch applies fine but mysql80-server fails to compile here:

/wrkdirs/usr/ports/databases/mysql80-server/work/mysql-8.0.28/storage/innobase/ut/crc32.cc:501:3: error: use of undeclared identifier 
'__crc32cb'
  __crc32cb(crc, data);
  ^
/wrkdirs/usr/ports/databases/mysql80-server/work/mysql-8.0.28/storage/innobase/ut/crc32.cc:505:3: error: use of undeclared identifier 
'__crc32ch'
  __crc32ch(crc, data);
  ^
/wrkdirs/usr/ports/databases/mysql80-server/work/mysql-8.0.28/storage/innobase/ut/crc32.cc:509:3: error: use of undeclared identifier 
'__crc32cw'
  __crc32cw(crc, data);
  ^
/wrkdirs/usr/ports/databases/mysql80-server/work/mysql-8.0.28/storage/innobase/ut/crc32.cc:513:3: error: use of undeclared identifier 
'__crc32cd'
  __crc32cd(crc, data);
  ^
4 errors generated.
*** [storage/innobase/CMakeFiles/innodb_zipdecompress.dir/ut/crc32.cc.o] Error code 1
Comment 10 tech-lists 2022-02-15 02:12:32 UTC
seems adding "CFLAGS+= -march=armv8.1-a+crc+crypto" to the poudriere's make.conf had no effect, however adding it to its Makefile (thank you, Daniel) after applying the patch allowed it to build
Comment 11 Daniel O'Connor 2022-02-15 10:57:04 UTC
Created attachment 231833 [details]
Smaller diff for CRC32/Crypto fixes

This is a much smaller patch that still compiles cleanly.
Comment 12 Daniel O'Connor 2022-02-15 10:57:39 UTC
Created attachment 231834 [details]
Patch port makefile to enable crc & crypto instructions
Comment 13 commit-hook freebsd_committer freebsd_triage 2022-02-27 17:20:51 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/ports/commit/?id=b042a3d8ca08c0da0f423e0afe49c8e512f5d8ed

commit b042a3d8ca08c0da0f423e0afe49c8e512f5d8ed
Author:     Daniel O'Connor <darius@dons.net.au>
AuthorDate: 2022-02-27 17:08:02 +0000
Commit:     Mikael Urankar <mikael@FreeBSD.org>
CommitDate: 2022-02-27 17:12:20 +0000

    databases/mysql80-server: Fix build on aarch64

      - Replace getauxval with the FreeBSD variant (elf_aux_info)
      - ifdef some gcc'ism and pass the correct cflags for clang

    Approved by:    portmgr (build fix blanket)
    PR:             261660

 databases/mysql80-server/Makefile                  |   1 +
 .../files/patch-storage_innobase_ut_crc32.cc (new) | 114 +++++++++++++++++++++
 2 files changed, 115 insertions(+)
Comment 14 Mikael Urankar freebsd_committer freebsd_triage 2022-02-27 17:28:50 UTC
Thank!