Bug 272900 - Mk/Uses/cpufeatures.mk - Framework helper for detecting CPU features/instructions
Summary: Mk/Uses/cpufeatures.mk - Framework helper for detecting CPU features/instruct...
Status: New
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Ports Framework (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Only Me
Assignee: Port Management Team
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2023-08-02 20:06 UTC by Daniel Engberg
Modified: 2023-08-29 19:53 UTC (History)
8 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Daniel Engberg freebsd_committer freebsd_triage 2023-08-02 20:06:25 UTC
Some upstream projects have specific optimized code paths which is great for peformance however many times they're are not detected during run-time but instead hardcoded and selected during configure/build stage. Currently we have no way of detecting features but instead need to rely on OPTIONS which is a pain and can lead to incorrect choices and/or broken binaries if wrong options are selected.

Having a quick look grep returns the following and I'm quite sure we have more potential users however maintainers haven't bothered because lack of framework functionality.

grep -ir AVX --include Makefile /usr/ports/ | grep OPTIONS_DEFINE | wc -l
23
grep -ir SSE --include Makefile /usr/ports/ | grep OPTIONS_DEFINE | wc -l
63
grep -ir NEON --include Makefile /usr/ports/ | grep OPTIONS_DEFINE | wc -l
7
grep -ir VSX --include Makefile /usr/ports/ | grep OPTIONS_DEFINE | wc -l
2

There are at least two way of getting what features the compiler defines:

clang -v -march=native -E - </dev/null
echo | clang -E - -march=native -###

I'm not entirely sure on how to implement this though but I think most one viable solution would to be adding a bunch of variables under a specific temple and -march=native would follow CPUTYPE.

Target output looks like this:
-target-feature +crc32 -target-feature -fma4

CPU_FEAT_<FEATURE> crc32 would be CPU_FEAT_CRC32 and fma4 not initialized at all since it's being disabled etc

Any ideas and/or input?
Comment 1 Piotr Kubaj freebsd_committer freebsd_triage 2023-08-03 16:42:48 UTC
I added specific powerpc*-related entries to MACHINE_CPU in https://cgit.freebsd.org/src/tree/share/mk/bsd.cpu.mk#n279. Then it can be used like: https://cgit.freebsd.org/ports/tree/graphics/opencv/Makefile#n219

IMO adding a ports-specific Mk/Uses/cpufeatures.mk is redundant, we already have MACHINE_CPU which covers this. It's just that ARM and RISCV would need MACHINE_CPU properly set and then ports can start using it. Obviously it wouldn't work on 12 and 13.
Comment 2 Daniel Engberg freebsd_committer freebsd_triage 2023-08-03 16:51:46 UTC
Interesting, downside with that approach is that it relies on manual entries and only covers a subset at best?
Comment 3 Piotr Kubaj freebsd_committer freebsd_triage 2023-08-03 17:14:22 UTC
Well, it works for POWER, because we only have a few CPU's and POWER supports only a couple of SIMD standards (no AVX512 mess).
Comment 4 Robert Clausecker freebsd_committer freebsd_triage 2023-08-04 22:14:34 UTC
If you add this, I would like to have

 - a macro BYTEORDER which evaluates to 1234 or 4321 depending on arch
 - a macro WORDSIZE which evaluates to 32 or 64 depending on arch
 - macros {NOT,ONLY}_FOR_{32,64}{BE,LE} to conditionally enable/disable
   ports that are known to require bit/little/32 bit/64 bit architectures
Comment 5 Piotr Kubaj freebsd_committer freebsd_triage 2023-08-04 22:36:53 UTC
(In reply to Robert Clausecker from comment #4)
The first two were recently added to bsd.cpu.mk: https://cgit.freebsd.org/src/tree/share/mk/bsd.cpu.mk#n390
Comment 6 Robert Clausecker freebsd_committer freebsd_triage 2023-08-04 22:46:36 UTC
(In reply to Piotr Kubaj from comment #5)

Cool!  Though I don't think bsd.cpu.mk is usable from Ports makefiles.
Comment 7 Daniel Engberg freebsd_committer freebsd_triage 2023-08-05 06:32:27 UTC
Would parsing output be a bad idea compared to maintaining static lists? In that regard I think we can assume that the compiler get things right or at least closer than maintaining a list by hand?
Comment 8 Piotr Kubaj freebsd_committer freebsd_triage 2023-08-05 09:12:17 UTC
(In reply to Robert Clausecker from comment #6)
It is, MACHINE_ABI is usable.

(In reply to Daniel Engberg from comment #7)
For architectures that have more SIMD variations and more CPU models, generated lists would probably be better, but doesn't it depend on -march=native? POWER doesn't support -march=native, so a manually maintained list would be necessary. I'm also not sure about RISCV.
Comment 9 Daniel Engberg freebsd_committer freebsd_triage 2023-08-05 09:38:13 UTC
(In reply to Piotr Kubaj from comment #8)
I think you misunderstand or I wasn't clear about it, I don't want to use -native specifically. 

"and -march=native would follow CPUTYPE" --> -march=${CPUTYPE}

Not sure if we should use a list like we're doing right now or generate variables using a documented "protected" naming schema.

Like HAVE_CPU_FEAT_${TARGET_FEATURE} or HAVE_CPU_INSTR_${TARGET_FEATURE} where it would be extracted from compiler output. 
Example: .... -target-feature +mmx -target-feature +avx2 ...
HAVE_CPU_FEAT_MMX , HAVE_CPU_FEAT_AVX2

Best regards,
Daniel