Bug 257622

Summary: textproc/flex: respect CC ?
Product: Ports & Packages Reporter: Evgenii Khramtsov <2khramtsov>
Component: Individual Port(s)Assignee: Matthias Andree <mandree>
Status: Closed FIXED    
Severity: Affects Only Me Keywords: patch
Priority: --- Flags: mandree: maintainer-feedback+
Version: Latest   
Hardware: Any   
OS: Any   
See Also: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=230018
Attachments:
Description Flags
v1 (use "git am") mandree: maintainer-approval-

Description Evgenii Khramtsov 2021-08-04 16:36:38 UTC
Created attachment 226948 [details]
v1 (use "git am")

Unbreak the build when the base toolchain (/usr/bin/cc) does not exist,
and CC is provided via inclusion of /usr/local/share/toolchains.

Note, this is a quick *workaround*, and seems to be similar to
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=257566
BINARY_ALIAS can also be (ab)used to fix this.

ENOTIME to find the not-workaround solution, but better to leave the workaround here in this PR.
Comment 1 Matthias Andree freebsd_committer freebsd_triage 2021-08-04 16:49:40 UTC
Evgeniy, 

thanks for the patch, but can you show me how to reproduce the "inclusion of /usr/local/share/toolchains" part?

I tried renaming cc to cc.off and the naive "make CROSS_TOOLCHAIN=llvm10" which failed with "make: "/usr/ports/Mk/bsd.port.mk" line 1088: Unknown directive "export"" and tripped over the .export.env here:

# only bmake support the below
STRIPBIN=   ${STRIP_CMD}
.export.env STRIPBIN
.endif

Also, normally, a GNU configure would pick up CC from the environment, and make CC=clang10 succeeds if cc is not present.

TIA
Comment 2 Evgenii Khramtsov 2021-08-05 01:28:10 UTC
(In reply to Matthias Andree from comment #1)

Sure:

I have these in src.conf:

CROSS_TOOLCHAIN=llvm12
WITHOUT_CLANG=yes
WITHOUT_CLANG_BOOTSTRAP=yes
# XXX: use from ports later
# WITHOUT_ELFTOOLCHAIN_BOOTSTRAP=yes
WITHOUT_LLD=yes
WITHOUT_LLD_BOOTSTRAP=yes
WITHOUT_LLDB=yes
WITHOUT_LLVM_COV=yes
+ base patch https://github.com/DankBSD/base/commit/21b52844f6e4e8204b15c97f3b48ca98af43c598

And these ports patches:
https://codeberg.org/ei/ports/commit/ee7bab09690cd4781b60d18d540c5c9e6a7d3c85
https://codeberg.org/ei/ports/commit/ba311d3eb4e7504464eaed6bd9a6508dc5cc9234
https://codeberg.org/ei/ports/commit/5070fd2a4932ab9d6adff0c3a5944d4adb055b1f

I build in a poudriere jail that was built using src.conf and the base patch above, with ports tree that has three ports patches above. Note, make delete-old
will not delete /usr/bin/cc, so you can't use an upgrade an existing jail.

This might be somewhat hacky, but I have the $CC, etc, defined, and build almost all (~640) desktop related ports without issues, except for two or three autoconf ports (many other ports using autoconf build fine). Any other out of tree ports patches are in ei/ports.
Comment 3 Evgenii Khramtsov 2021-08-05 01:40:32 UTC
(In reply to Evgeniy Khramtsov from comment #2)

I guess you can also check with aarch64 qemu-user-static jail with native-xtoolchain instead.
Comment 4 Matthias Andree freebsd_committer freebsd_triage 2021-08-05 09:51:59 UTC
Sorry, but if I would have to tweak world first, I don't think this is an issue for existing official src/ports trees.

Let me know if official supported releases break.
Comment 5 Evgenii Khramtsov 2021-08-05 10:41:03 UTC
(In reply to Matthias Andree from comment #4)

Steps to reproduce using official src/ports and the official native-xtoolchain jail built by poudriere using "-x" flag:

1. zfs rename bsd/poudriere/jails/130-aarch64@clean bsd/poudriere/jails/130-aarch64@bloat
2. mv /usr/local/poudriere/jails/130-aarch64/usr/bin/cc /usr/local/poudriere/jails/130-aarch64/usr/bin/bloat
3. poudriere testport -j 130-aarch64 -p main textproc/flex

(poudriere outputs configured native-xtoolchain in log)
[...]
CPP=/nxb-bin/usr/bin/cpp
CXX=/nxb-bin/usr/bin/c++
NM=/nxb-bin/usr/bin/nm
LD=/nxb-bin/usr/bin/ld
OBJCOPY=/nxb-bin/usr/bin/objcopy
SIZE=/nxb-bin/usr/bin/size
STRIPBIN=/nxb-bin/usr/bin/strip
SED=/nxb-bin/usr/bin/sed
RANLIB=/nxb-bin/usr/bin/ranlib
YACC=/nxb-bin/usr/bin/yacc
MAKE=/nxb-bin/usr/bin/make
STRINGS=/nxb-bin/usr/bin/strings
AWK=/nxb-bin/usr/bin/awk
FLEX=/nxb-bin/usr/bin/flex 
[...]

checking for gcc... (cached) /nxb-bin/usr/bin/cc
checking whether we are using the GNU C compiler... (cached) yes
checking whether /nxb-bin/usr/bin/cc accepts -g... (cached) yes
checking for /nxb-bin/usr/bin/cc option to accept ISO C89... (cached) none needed
checking whether /nxb-bin/usr/bin/cc understands -c and -o together... (cached) yes
checking for gcc... no
checking for cc... no
checking for cl.exe... no
configure: error: in `/wrkdirs/usr/ports/textproc/flex/work/flex-2.6.4':
configure: error: no acceptable C compiler found in $PATH
See `config.log' for more details
===>  Script "configure" failed unexpectedly.
Please report the problem to mandree@FreeBSD.org [maintainer] and attach the
"/wrkdirs/usr/ports/textproc/flex/work/flex-2.6.4/config.log" including the
output of the failure of your make command. Also, it might be a good idea to
provide an overview of all packages installed on your system (e.g. a
/usr/local/sbin/pkg-static info -g -Ea).
*** Error code 1

My setup exposes ports not respecting CC well:
https://docs.freebsd.org/en/books/porters-handbook/porting-dads/#dads-cc
Comment 6 Evgenii Khramtsov 2021-08-05 10:44:05 UTC
(In reply to Evgeniy Khramtsov from comment #5)

Missed line while pasting from poudriere log:

---Begin make.nxb.conf---
CC=/nxb-bin/usr/bin/cc
Comment 7 Evgenii Khramtsov 2021-08-05 10:48:37 UTC
(In reply to Evgeniy Khramtsov from comment #5)

The step before testport would be to snapshot 130-aarch64 to @clean
Sorry for being unfocused.

> 3. poudriere testport -j 130-aarch64 -p main textproc/flex
Comment 8 Matthias Andree freebsd_committer freebsd_triage 2021-08-05 11:04:42 UTC
Evgeniy,

I can reproduce this now with my 12.x ARM64 (aarch64) jail under qemu. 

The thing is that flex has special support for cross-compiling because it needs compilers that generate code both for the host (CC_FOR_BUILD) and the target (CC). These are not the same, and if you take away the host compiler, by moving cc away, the build breaks.

I'll look around a bit but I'm not sure we're going towards the same goal yet. 

I need to understand what FreeBSD means with XCC and CC, I am not familiar with cross-building on FreeBSD. Pointers (deep links) to documentation solicited.
Comment 9 Evgenii Khramtsov 2021-08-05 11:41:54 UTC
(In reply to Matthias Andree from comment #8)

foo@bar:/usr/ports $ rg 'CC_FOR_BUILD' .

reveals many ports (e.g. ports-mgmt/pkg, x11-toolkits/gtk30) passing CC_FOR_BUILD to respect CC. Note, I am not a GNU tools expert, I can't help more.

Patching configure is not needed, CC_FOR_BUILD can be passed via port Makefile.
Comment 10 Matthias Andree freebsd_committer freebsd_triage 2021-08-05 11:43:58 UTC
Yeah, I think we can go that way.
Comment 11 Matthias Andree freebsd_committer freebsd_triage 2021-08-05 12:07:42 UTC
Comment on attachment 226948 [details]
v1 (use "git am")

let's move this into the Makefile where it remains visible.
Comment 12 commit-hook freebsd_committer freebsd_triage 2021-08-05 12:08:22 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/ports/commit/?id=828ef20174745ff612b26c16e17a40f458854ab2

commit 828ef20174745ff612b26c16e17a40f458854ab2
Author:     Matthias Andree <mandree@FreeBSD.org>
AuthorDate: 2021-08-05 12:05:08 +0000
Commit:     Matthias Andree <mandree@FreeBSD.org>
CommitDate: 2021-08-05 12:07:23 +0000

    textproc/flex: set *_FOR_BUILD variables...

    ...to unbreak the build when ax_prog_cc_for_build.m4 hunts a host
    compiler in path, pushing CC aside.

    PR:             257622
    Reported by:    Evgeniy Khramtsov

 textproc/flex/Makefile | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)