Bug 272900

Summary:	Mk/Uses/cpufeatures.mk - Framework helper for detecting CPU features/instructions
Product:	Ports & Packages	Reporter:	Daniel Engberg <diizzy>
Component:	Ports Framework	Assignee:	Port Management Team <portmgr>
Status:	New ---
Severity:	Affects Only Me	CC:	agh, bsd, emaste, fuz, mandree, pkubaj, ports-bugs, yuri, zirias
Priority:	---
Version:	Latest
Hardware:	Any
OS:	Any

Description Daniel Engberg freebsd_committer

2023-08-02 20:06:25 UTC

Some upstream projects have specific optimized code paths which is great for peformance however many times they're are not detected during run-time but instead hardcoded and selected during configure/build stage. Currently we have no way of detecting features but instead need to rely on OPTIONS which is a pain and can lead to incorrect choices and/or broken binaries if wrong options are selected.

Having a quick look grep returns the following and I'm quite sure we have more potential users however maintainers haven't bothered because lack of framework functionality.

grep -ir AVX --include Makefile /usr/ports/ | grep OPTIONS_DEFINE | wc -l
23
grep -ir SSE --include Makefile /usr/ports/ | grep OPTIONS_DEFINE | wc -l
63
grep -ir NEON --include Makefile /usr/ports/ | grep OPTIONS_DEFINE | wc -l
7
grep -ir VSX --include Makefile /usr/ports/ | grep OPTIONS_DEFINE | wc -l
2

There are at least two way of getting what features the compiler defines:

clang -v -march=native -E - </dev/null
echo | clang -E - -march=native -###

I'm not entirely sure on how to implement this though but I think most one viable solution would to be adding a bunch of variables under a specific temple and -march=native would follow CPUTYPE.

Target output looks like this:
-target-feature +crc32 -target-feature -fma4

CPU_FEAT_<FEATURE> crc32 would be CPU_FEAT_CRC32 and fma4 not initialized at all since it's being disabled etc

Any ideas and/or input?

Comment 1 Piotr Kubaj freebsd_committer

2023-08-03 16:42:48 UTC

I added specific powerpc*-related entries to MACHINE_CPU in https://cgit.freebsd.org/src/tree/share/mk/bsd.cpu.mk#n279. Then it can be used like: https://cgit.freebsd.org/ports/tree/graphics/opencv/Makefile#n219

IMO adding a ports-specific Mk/Uses/cpufeatures.mk is redundant, we already have MACHINE_CPU which covers this. It's just that ARM and RISCV would need MACHINE_CPU properly set and then ports can start using it. Obviously it wouldn't work on 12 and 13.

Comment 2 Daniel Engberg freebsd_committer

2023-08-03 16:51:46 UTC

Interesting, downside with that approach is that it relies on manual entries and only covers a subset at best?

Comment 3 Piotr Kubaj freebsd_committer

2023-08-03 17:14:22 UTC

Well, it works for POWER, because we only have a few CPU's and POWER supports only a couple of SIMD standards (no AVX512 mess).

Comment 4 Robert Clausecker freebsd_committer

2023-08-04 22:14:34 UTC

If you add this, I would like to have

 - a macro BYTEORDER which evaluates to 1234 or 4321 depending on arch
 - a macro WORDSIZE which evaluates to 32 or 64 depending on arch
 - macros {NOT,ONLY}_FOR_{32,64}{BE,LE} to conditionally enable/disable
   ports that are known to require bit/little/32 bit/64 bit architectures

Comment 5 Piotr Kubaj freebsd_committer

2023-08-04 22:36:53 UTC

(In reply to Robert Clausecker from comment #4)
The first two were recently added to bsd.cpu.mk: https://cgit.freebsd.org/src/tree/share/mk/bsd.cpu.mk#n390

Comment 6 Robert Clausecker freebsd_committer

2023-08-04 22:46:36 UTC

(In reply to Piotr Kubaj from comment #5)

Cool!  Though I don't think bsd.cpu.mk is usable from Ports makefiles.

Comment 7 Daniel Engberg freebsd_committer

2023-08-05 06:32:27 UTC

Would parsing output be a bad idea compared to maintaining static lists? In that regard I think we can assume that the compiler get things right or at least closer than maintaining a list by hand?

Comment 8 Piotr Kubaj freebsd_committer

2023-08-05 09:12:17 UTC

(In reply to Robert Clausecker from comment #6)
It is, MACHINE_ABI is usable.

(In reply to Daniel Engberg from comment #7)
For architectures that have more SIMD variations and more CPU models, generated lists would probably be better, but doesn't it depend on -march=native? POWER doesn't support -march=native, so a manually maintained list would be necessary. I'm also not sure about RISCV.

Comment 9 Daniel Engberg freebsd_committer

2023-08-05 09:38:13 UTC

(In reply to Piotr Kubaj from comment #8)
I think you misunderstand or I wasn't clear about it, I don't want to use -native specifically. 

"and -march=native would follow CPUTYPE" --> -march=${CPUTYPE}

Not sure if we should use a list like we're doing right now or generate variables using a documented "protected" naming schema.

Like HAVE_CPU_FEAT_${TARGET_FEATURE} or HAVE_CPU_INSTR_${TARGET_FEATURE} where it would be extracted from compiler output. 
Example: .... -target-feature +mmx -target-feature +avx2 ...
HAVE_CPU_FEAT_MMX , HAVE_CPU_FEAT_AVX2

Best regards,
Daniel