See comments in https://gitweb.gentoo.org/repo/gentoo.git/tree/sys-libs/zlib-ng/zlib-ng-2.2.4.ebuild As much as https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=272900 would help and fix the issue you probably need to set some baseline options (like Go lang compiler does) and remove usage of -march completely.
https://github.com/trifectatechfoundation/zlib-rs might be of interest simply because it seems to handle this much better however compatibility needs to be verified
I know about this, but it seems to me that you don't quite understand how it's actually done there. :-) "-march" and " add_feature_info(WITH_AVX2 WITH_AVX2 "Build with AVX2") add_feature_info(WITH_AVX512 WITH_AVX512 "Build with AVX512") add_feature_info(WITH_AVX512VNNI WITH_AVX512VNNI "Build with AVX512 VNNI") add_feature_info(WITH_SSE2 WITH_SSE2 "Build with SSE2") add_feature_info(WITH_SSSE3 WITH_SSSE3 "Build with SSSE3") add_feature_info(WITH_SSE42 WITH_SSE42 "Build with SSE42") add_feature_info(WITH_PCLMULQDQ WITH_PCLMULQDQ "Build with PCLMULQDQ") add_feature_info(WITH_VPCLMULQDQ WITH_VPCLMULQDQ "Build with VPCLMULQDQ") " are different things. "-march" is a compiler option that allows additional instructions and optimizations to be used where the compiler wants them. "WITH_xxx" are for use hardcoded intrinsics. Also it detects in runtime supported instructions via cpuid instruction in arch/x86/x86_features.c. If you want to allow hardcoded intrinsics to be used by those who build the port themselves, then we just need to add all those options to the port. For example: https://cgit.freebsd.org/ports/tree/graphics/khronos-texture/Makefile#n20
(In reply to Daniel Engberg from comment #1) I'm not interested in ports that use rust.
Created attachment 258938 [details] v1 Need testing on arm*, powerpc*, riscv64.
"make configure" with "-march=core2" and with the patch: -- Performing Test HAVE_XSAVE_INTRIN -- Performing Test HAVE_XSAVE_INTRIN - Success -- Performing Test HAVE_SSE2_INTRIN -- Performing Test HAVE_SSE2_INTRIN - Success -- Performing Test HAVE_SSSE3_INTRIN -- Performing Test HAVE_SSSE3_INTRIN - Success -- Architecture-specific source files: arch/x86/x86_features.c;arch/x86/chunkset_sse2.c;arch/x86/compare256_sse2.c;arch/x86/slide_hash_sse2.c;arch/x86/adler32_ssse3.c;arch/x86/chunkset_ssse3.c -- The following features have been enabled: * CMAKE_BUILD_TYPE, Build type: Release (selected) * XSAVE, Support XSAVE intrinsics using "-mxsave" * SSSE3_ADLER32, Support SSSE3-accelerated adler32, using "-mssse3" * WITH_GZFILEOP, Compile with support for gzFile related functions * WITH_SANITIZER, Enable sanitizer support * WITH_GTEST, Build gtest_zlib * WITH_OPTIM, Build with optimisation * WITH_NEW_STRATEGIES, Use new strategies * WITH_RUNTIME_CPU_DETECTION, Build with runtime CPU detection * WITH_SSE2, Build with SSE2 * WITH_SSSE3, Build with SSSE3 -- The following features have been disabled: * ZLIB_SYMBOL_PREFIX, Publicly exported symbols DO NOT have a custom prefix * ZLIB_COMPAT, Compile with zlib compatible API * ZLIB_ENABLE_TESTS, Build test binaries * ZLIBNG_ENABLE_TESTS, Test zlib-ng specific API * WITH_FUZZERS, Build test/fuzz * WITH_BENCHMARKS, Build test/benchmarks * WITH_BENCHMARK_APPS, Build application benchmarks * WITH_NATIVE_INSTRUCTIONS, Instruct the compiler to use the full instruction set on this host (gcc/clang -march=native) * WITH_MAINTAINER_WARNINGS, Build with project maintainer warnings * WITH_CODE_COVERAGE, Enable code coverage reporting * WITH_INFLATE_STRICT, Build with strict inflate distance checking * WITH_INFLATE_ALLOW_INVALID_DIST, Build with zero fill for inflate invalid distances * WITH_AVX2, Build with AVX2 * WITH_AVX512, Build with AVX512 * WITH_AVX512VNNI, Build with AVX512 VNNI * WITH_SSE42, Build with SSE42 * WITH_PCLMULQDQ, Build with PCLMULQDQ * WITH_VPCLMULQDQ, Build with VPCLMULQDQ * INSTALL_UTILS, Copy minigzip and minideflate during install "make configure" with "-march=core2" and without the patch: -- Performing Test HAVE_XSAVE_INTRIN -- Performing Test HAVE_XSAVE_INTRIN - Success -- Performing Test HAVE_SSE2_INTRIN -- Performing Test HAVE_SSE2_INTRIN - Success -- Performing Test HAVE_SSSE3_INTRIN -- Performing Test HAVE_SSSE3_INTRIN - Success -- Performing Test HAVE_SSE42_INTRIN -- Performing Test HAVE_SSE42_INTRIN - Success -- Performing Test HAVE_PCLMULQDQ_INTRIN -- Performing Test HAVE_PCLMULQDQ_INTRIN - Success -- Performing Test HAVE_AVX2_INTRIN -- Performing Test HAVE_AVX2_INTRIN - Success -- Performing Test HAVE_CASCADE_LAKE -- Performing Test HAVE_CASCADE_LAKE - Success -- Performing Test HAVE_AVX512_INTRIN -- Performing Test HAVE_AVX512_INTRIN - Success -- Performing Test HAVE_AVX512VNNI_INTRIN -- Performing Test HAVE_AVX512VNNI_INTRIN - Success -- Performing Test HAVE_VPCLMULQDQ_INTRIN -- Performing Test HAVE_VPCLMULQDQ_INTRIN - Success -- Architecture-specific source files: arch/x86/x86_features.c;arch/x86/chunkset_sse2.c;arch/x86/compare256_sse2.c;arch/x86/slide_hash_sse2.c;arch/x86/adler32_ssse3.c;arch/x86/chunkset_ssse3.c;arch/x86/adler32_sse42.c;arch/x86/crc32_pclmulqdq.c;arch/x86/slide_hash_avx2.c;arch/x86/chunkset_avx2.c;arch/x86/compare256_avx2.c;arch/x86/adler32_avx2.c;arch/x86/adler32_avx512.c;arch/x86/chunkset_avx512.c;arch/x86/adler32_avx512_vnni.c;arch/x86/crc32_vpclmulqdq.c -- The following features have been enabled: * CMAKE_BUILD_TYPE, Build type: Release (selected) * XSAVE, Support XSAVE intrinsics using "-mxsave" * SSSE3_ADLER32, Support SSSE3-accelerated adler32, using "-mssse3" * SSE42_CRC, Support SSE4.2 optimized adler32 hash generation, using "-msse4.2" * PCLMUL_CRC, Support CRC hash generation using PCLMULQDQ, using "-msse4.2 -mpclmul" * AVX2_SLIDEHASH, Support AVX2 optimized slide_hash, using "-mavx2 -mbmi2" * AVX2_CHUNKSET, Support AVX2 optimized chunkset, using "-mavx2 -mbmi2" * AVX2_COMPARE256, Support AVX2 optimized compare256, using "-mavx2 -mbmi2" * AVX2_ADLER32, Support AVX2-accelerated adler32, using "-mavx2 -mbmi2" * AVX512_ADLER32, Support AVX512-accelerated adler32, using "-mavx512f -mavx512dq -mavx512bw -mavx512vl -mbmi2 -mtune=cascadelake" * AVX512_CHUNKSET, Support AVX512 optimized chunkset, using "-mavx512f -mavx512dq -mavx512bw -mavx512vl -mbmi2 -mtune=cascadelake" * AVX512VNNI_ADLER32, Support AVX512VNNI adler32, using "-mavx512f -mavx512dq -mavx512bw -mavx512vl -mavx512vnni -mbmi2 -mtune=cascadelake" * VPCLMUL_CRC, Support CRC hash generation using VPCLMULQDQ, using "-mpclmul -mvpclmulqdq -mavx512f -mavx512f -mavx512dq -mavx512bw -mavx512vl -mbmi2 -mtune=cascadelake" * WITH_GZFILEOP, Compile with support for gzFile related functions * WITH_SANITIZER, Enable sanitizer support * WITH_GTEST, Build gtest_zlib * WITH_OPTIM, Build with optimisation * WITH_NEW_STRATEGIES, Use new strategies * WITH_RUNTIME_CPU_DETECTION, Build with runtime CPU detection * WITH_AVX2, Build with AVX2 * WITH_AVX512, Build with AVX512 * WITH_AVX512VNNI, Build with AVX512 VNNI * WITH_SSE2, Build with SSE2 * WITH_SSSE3, Build with SSSE3 * WITH_SSE42, Build with SSE42 * WITH_PCLMULQDQ, Build with PCLMULQDQ * WITH_VPCLMULQDQ, Build with VPCLMULQDQ -- The following features have been disabled: * ZLIB_SYMBOL_PREFIX, Publicly exported symbols DO NOT have a custom prefix * ZLIB_COMPAT, Compile with zlib compatible API * ZLIB_ENABLE_TESTS, Build test binaries * ZLIBNG_ENABLE_TESTS, Test zlib-ng specific API * WITH_FUZZERS, Build test/fuzz * WITH_BENCHMARKS, Build test/benchmarks * WITH_BENCHMARK_APPS, Build application benchmarks * WITH_NATIVE_INSTRUCTIONS, Instruct the compiler to use the full instruction set on this host (gcc/clang -march=native) * WITH_MAINTAINER_WARNINGS, Build with project maintainer warnings * WITH_CODE_COVERAGE, Enable code coverage reporting * WITH_INFLATE_STRICT, Build with strict inflate distance checking * WITH_INFLATE_ALLOW_INVALID_DIST, Build with zero fill for inflate invalid distances * INSTALL_UTILS, Copy minigzip and minideflate during install It looks like it builds all possible optimizations by default, but at runtime it determines which ones to use depending on the instructions the CPU supports. So my patch looks useless.
A much simplier way would be to enable WITH_NATIVE_INSTRUCTIONS and patch out --march=native in CMakeLists.txt when CPUTYPE is defined however that doesn't seem to work as cmake/detect-intrinsics.cmake doesn't seem to test anything when WITH_NATIVE_INSTRUCTIONS is enabled so I don't see how that option is supposed to work. Apart from that -mtune=*lake should probably target baseline-x86-64-v4 rather than an Intel-specific CPUs for compatibility reasons.
-march=native should be disabled in any case, as it causes unportable binaries to be produced.
No one said anything else?
(In reply to Vladimir Druzenko from comment #4) The TEST option should be changed so "make test" always works. Builds and tests fine on armv7 and aarch64 FreeBSD 15-CURRENT.
(In reply to Daniel Engberg from comment #8) What else can be said here except that native is broken - it doesn't work properly, and it's better not to use it.
(In reply to Robert Clausecker from comment #9) Thanks! But I think this patch is nonsensical: it disables runtime detection and the use of all SIMD that are not supported by the base architecture (build binary packages with default options), for example on amd64 this is everything newer than SSE2. That is, binary packages are deprived of these optimizations and become slower on all systems starting from about Core 2 Duo.
(In reply to Robert Clausecker from comment #9) Can you please show example? Or even patch in comments… :-)
(In reply to Vladimir Druzenko from comment #12) For example (untested): diff --git a/archivers/zlib-ng/Makefile b/archivers/zlib-ng/Makefile index 35861f958531..b57d78999b86 100644 --- a/archivers/zlib-ng/Makefile +++ b/archivers/zlib-ng/Makefile @@ -10,14 +10,16 @@ WWW= https://github.com/zlib-ng/zlib-ng/ LICENSE= ZLIB LICENSE_FILE= ${WRKSRC}/LICENSE.md -USES= cmake:${TESTING} compiler:c11 +TEST_DEPENDS= googletest>=0:devel/googletest + +USES= cmake:testing compiler:c11 USE_GITHUB= yes USE_LDCONFIG= yes PLIST_SUB= SHLIB_VER=${DISTVERSION} # The names and order of the options are the same as in the ${PORTNAME}'s CMakeLists.txt -OPTIONS_DEFINE= OPTIM TEST +OPTIONS_DEFINE= OPTIM OPTIONS_DEFAULT= OPTIM OPTIONS_DEFAULT_aarch64= ACLE NEON OPTIONS_DEFAULT_amd64= SSE2 @@ -98,7 +100,7 @@ VPCLMULQDQ_IMPLIES= PCLMULQDQ AVX512 VPCLMULQDQ_CMAKE_BOOL= WITH_VPCLMULQDQ TEST_BUILD_DEPENDS= googletest>=0:devel/googletest -TEST_CMAKE_BOOL= ZLIB_ENABLE_TESTS ZLIBNG_ENABLE_TESTS +CMAKE_TESTING_ON= ZLIB_ENABLE_TESTS ZLIBNG_ENABLE_TESTS # error: use of undeclared identifier 'isascii' TEST_CFLAGS= -D__XSI_VISIBLE TEST_VARS= TESTING=testing ------------ The CMAKE_TESTING_* macros are like the normal ones but only apply during testing.
(In reply to Robert Clausecker from comment #13) 1. How to transfer -D__XSI_VISIBLE to "CMAKE_TESTING"? I tried: CMAKE_TESTING_ARGS, CMAKE_TESTING_SETENV - doesn't work. Only "CFLAGS+=-D__XSI_VISIBLE" work. 2. "TEST_DEPENDS= googletest>=0:devel/googletest" doesn't work. Work "BUILD_DEPENDS= googletest>=0:devel/googletest" 3. Number of "steps" during default build increased from 99 to 150. PORTNAME= zlib-ng DISTVERSION= 2.2.4 CATEGORIES= archivers MAINTAINER= vvd@FreeBSD.org COMMENT= Fork of the zlib data compression library WWW= https://github.com/zlib-ng/zlib-ng/ LICENSE= ZLIB LICENSE_FILE= ${WRKSRC}/LICENSE.md BUILD_DEPENDS= googletest>=0:devel/googletest USES= cmake:testing compiler:c11 USE_GITHUB= yes USE_LDCONFIG= yes # error: use of undeclared identifier 'isascii' CFLAGS+= -D__XSI_VISIBLE CMAKE_TESTING_ON= ZLIB_ENABLE_TESTS ZLIBNG_ENABLE_TESTS PLIST_SUB= SHLIB_VER=${DISTVERSION} OPTIONS_DEFINE= SIMD OPTIONS_DEFAULT= SIMD SIMD_CMAKE_BOOL= WITH_OPTIM .include <bsd.port.mk>
(In reply to Vladimir Druzenko from comment #14) Hello Vladimir, > 1. How to transfer -D__XSI_VISIBLE to "CMAKE_TESTING"? If it works with CFLAGS, just keep it there. > 2. "TEST_DEPENDS= googletest>=0:devel/googletest" doesn't work. > Work "BUILD_DEPENDS= googletest>=0:devel/googletest" That's unfortunate, but then you'll need to just keep it in BUILD_DEPENDS. > 3. Number of "steps" during default build increased from 99 to 150. Maybe it also needs CMAKE_OFF= ZLIB_ENABLE_TESTS ZLIBNG_ENABLE_TESTS
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/ports/commit/?id=18d66e2f3d9741d4aaceba7560a2b96f105c56e4 commit 18d66e2f3d9741d4aaceba7560a2b96f105c56e4 Author: Vladimir Druzenko <vvd@FreeBSD.org> AuthorDate: 2025-05-03 14:11:44 +0000 Commit: Vladimir Druzenko <vvd@FreeBSD.org> CommitDate: 2025-05-03 14:11:44 +0000 archivers/zlib-ng: Transfrom option TEST to support of "make test" PR: 285597 archivers/zlib-ng/Makefile | 17 ++++++++++------- 1 file changed, 10 insertions(+), 7 deletions(-)