Bug 210993 - WITH_META_MODE cross builds have "Exec format error" problems when certain areas attempt a rebuild
Summary: WITH_META_MODE cross builds have "Exec format error" problems when certain ar...
Status: Closed Overcome By Events
Alias: None
Product: Base System
Classification: Unclassified
Component: misc (show other bugs)
Version: CURRENT
Hardware: Any Any
: --- Affects Only Me
Assignee: freebsd-bugs (Nobody)
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2016-07-10 23:01 UTC by Mark Millard
Modified: 2018-08-12 15:35 UTC (History)
1 user (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Mark Millard 2016-07-10 23:01:53 UTC
For using WITH_META_MODE=yes again for going from -r302331 to -r302412 via cross builds amd64 -> armmv6 or powerpc64:

The build stopped with:

--- init_keytry.h ---
sh: ./make_keys: Exec format error
*** [init_keytry.h] Error code 126


Using a worldclean and then separately a buildworld was sufficient to get around the problem.

Past exchanges also reported:

> --- proctab.c --- 
> sh: ./maketab: Exec format error 
> *** [proctab.c] Error code 126 

But this only happened if one got by the make_keys one. I've not seen this one recently but the worldclean may be avoiding it too.

I only remember the two examples [make_keys and maketab] being found as having the issue but there could be more as far as I know.

At one time at least the maketab one was believed fixed after -r301884 (2016-Jun-14 or so).

I expect "Exec format error" is an issue in certain areas whenever TARGET_ARCH is incompatible with the host architecture. amd64 -> armv6/powerpc64 are just the examples from my environment. Prior activity ends up leaving the incompatible file(s) where later rebuilds try to use them on the host.

If it turns out that WITH_META_MODE=yes rebuilding a cross build does not execute make_keys or maketabs [or . . .?] then it works fine. At least is what I expect is the reason that some of my WITH_META_MODE=yes cross-rebuilds have had no problems.



It may be that UPDATING for 11.0-RELEASE and 11.0-STABLE and the like just will need notes something like the following for now:

"If you run into 'Exec format error' during a WITH_META_MODE=yes rebuild for a cross build (TARGET_ARCH=. . .) then you will need to do a make worldclean and then another buildworld (and possibly build kernel) for the TARGET_ARCH. An alternative is to not use WITH_META_MODE=yes ."
Comment 1 Mark Millard 2016-07-10 23:05:28 UTC
(In reply to Mark Millard from comment #0)

Below gives supporting context from an example failure.

In attempting to update my cross build (amd64 -> armv6 [or powerpc64]) from -r302331 to -r302412 it failed with [armv6 example]:

--- init_keytry.h ---
sh: ./make_keys: Exec format error
*** [init_keytry.h] Error code 126

make[4]: stopped in /usr/src/lib/ncurses/ncursesw
.ERROR_TARGET='init_keytry.h'
.ERROR_META_FILE='/usr/obj/clang/arm.armv6/usr/src/lib/ncurses/ncursesw/init_keytry.h.meta'
.MAKE.LEVEL='4'
MAKEFILE=''
.MAKE.MODE='meta missing-filemon=yes missing-meta=yes silent=yes verbose'
.CURDIR='/usr/src/lib/ncurses/ncursesw'
.MAKE='make'
.OBJDIR='/usr/obj/clang/arm.armv6/usr/src/lib/ncurses/ncursesw'
.TARGETS='all'
DESTDIR='/usr/obj/clang/arm.armv6/usr/src/tmp'
LD_LIBRARY_PATH=''
MACHINE='arm'
MACHINE_ARCH='armv6'
MAKEOBJDIRPREFIX='/usr/obj/clang/arm.armv6'
MAKESYSPATH='/usr/src/share/mk'
MAKE_VERSION='20160606'
PATH='/usr/obj/clang/arm.armv6/usr/src/tmp/legacy/usr/sbin:/usr/obj/clang/arm.armv6/usr/src/tmp/legacy/usr/bin:/usr/obj/clang/arm.armv6/usr/src/tmp/legacy/bin:/usr/obj/clang/arm.armv6/usr/src/tmp/usr/sbin:/usr/obj/clang/arm.armv6/usr/src/tmp/usr/bin:/sbin:/bin:/usr/sbin:/usr/bin'
SRCTOP='/usr/src'
OBJTOP='/usr/obj/clang/arm.armv6/usr/src'
.MAKE.MAKEFILES='/usr/src/share/mk/sys.mk /usr/src/share/mk/local.sys.env.mk /usr/src/share/mk/src.sys.env.mk /root/src.configs/src.conf.rpi2-clang-bootstrap.amd64-host /usr/src/share/mk/bsd.mkopt.mk /root/src.configs/make.conf /usr/src/share/mk/local.sys.mk /usr/src/share/mk/src.sys.mk /etc/src.conf /usr/src/lib/ncurses/ncursesw/Makefile /usr/src/lib/ncurses/ncursesw/../ncurses/Makefile /usr/src/share/mk/src.opts.mk /usr/src/share/mk/bsd.own.mk /usr/src/share/mk/bsd.opts.mk /usr/src/share/mk/bsd.cpu.mk /usr/src/share/mk/bsd.compiler.mk /usr/src/share/mk/bsd.compiler.mk /usr/src/lib/ncurses/ncursesw/../config.mk /usr/src/share/mk/bsd.lib.mk /usr/src/share/mk/bsd.init.mk /usr/src/share/mk/local.init.mk /usr/src/share/mk/src.init.mk /usr/src/lib/ncurses/ncursesw/../Makefile.inc /usr/src/lib/ncurses/ncursesw/../../Makefile.inc /usr/src/share/mk/bsd.libnames.mk /usr/src/share/mk/src.libnames.mk /usr/src/share/mk/bsd.symver.mk /usr/src/share/mk/bsd.nls.mk /usr/src/share/mk/bsd.files.mk /usr/src/share/mk/bsd.incs.mk /usr/src/share/mk/bsd.confs.mk /usr/src/share/mk/bsd.links.mk /usr/src/share/mk/bsd.dep.mk /usr/src/share/mk/bsd.clang-analyze.mk /usr/src/share/mk/bsd.obj.mk /usr/src/share/mk/bsd.subdir.mk /usr/src/share/mk/bsd.sys.mk'
.PATH='. /usr/src/lib/ncurses/ncursesw /usr/src/lib/ncurses/ncursesw/../ncurses /usr/src/lib/ncurses/ncursesw/../../../contrib/ncurses/include /usr/src/lib/ncurses/ncursesw/../../../contrib/ncurses/ncurses/base /usr/src/lib/ncurses/ncursesw/../../../contrib/ncurses/ncurses/tinfo /usr/src/lib/ncurses/ncursesw/../../../contrib/ncurses/ncurses/tty /usr/src/lib/ncurses/ncursesw/../../../contrib/ncurses/ncurses/widechar /usr/src/lib/ncurses/ncursesw/../../../contrib/ncurses/ncurses/trace /usr/src/lib/ncurses/ncursesw/../../../contrib/ncurses/man'
1 error

again.

This was based on:

# more ~/sys_build_scripts.amd64-host/make_rpi2_nodebug_clang_bootstrap-amd64-host.sh 
kldload -n filemon && \
script ~/sys_typescripts/typescript_make_rpi2_nodebug_clang_bootstrap-amd64-host-$(date +%Y-%m-%d:%H:%M:%S) \
env __MAKE_CONF="/root/src.configs/make.conf" SRC_ENV_CONF="/root/src.configs/src.conf.rpi2-clang-bootstrap.amd64-host" \
WITH_META_MODE=yes \
MAKEOBJDIRPREFIX="/usr/obj/clang" \
make $*

and. . .

# more ~/src.configs/src.conf.rpi2-clang-bootstrap.amd64-host 
TO_TYPE=armv6
#
KERNCONF=RPI2-NODBG
TARGET=arm
.if ${.MAKE.LEVEL} == 0
TARGET_ARCH=${TO_TYPE}
.export TARGET_ARCH
.endif
#
WITH_CROSS_COMPILER=
WITHOUT_SYSTEM_COMPILER=
#
#CPUTYPE=soft
WITH_LIBCPLUSPLUS=
WITH_BINUTILS_BOOTSTRAP=
WITH_CLANG_BOOTSTRAP=
WITH_CLANG=
WITH_CLANG_IS_CC=
WITH_CLANG_FULL=
WITH_CLANG_EXTRAS=
WITH_LLDB=
#
WITH_BOOT=
WITHOUT_LIB32=
WITHOUT_LIBSOFT=
#
WITHOUT_ELFTOOLCHAIN_BOOTSTRAP=
WITHOUT_GCC_BOOTSTRAP=
WITHOUT_GCC=
WITHOUT_GCC_IS_CC=
WITHOUT_GNUCXX=
#
NO_WERROR=
#WERROR=
MALLOC_PRODUCTION=
#
WITH_DEBUG_FILES=
#
XCFLAGS+= -march=armv7-a -mcpu=cortex-a7
XCXXFLAGS+= -march=armv7-a -mcpu=cortex-a7

make.conf was empty.

The earlier -r302331 cross build had WITH_LIBSOFT= in use. -r302412 is my first testing of WITHOUT_LIBSOFT= after rebuilding all ports to avoid libsoft. [Does not apply to the powerpc64 example.]
Comment 2 Mark Millard 2017-01-25 03:57:31 UTC
My jump from head -r312009 to -r312726 got another example of
this issue, a amd64 -> armv6 cross build (again). Likely
aarch64, powerpc, and powerpc64 will also have the problem.

[I also updated the Version field to CURRENT to 
match this new example. stable/11 and release/11.0
likely still have the problem as well.]

[I also changed from misc to bin as the Component, I
think I originally misclassified it.]
Comment 3 Mark Linimon freebsd_committer freebsd_triage 2018-08-12 14:38:50 UTC
Is this PR still relevant?
Comment 4 Mark Millard 2018-08-12 15:35:41 UTC
(In reply to Mark Linimon from comment #3)

I've not seen the problem in a very long time.

The example was supposedly fixed in -r301884,
but that is long before Comment 2's report.
(2016-Jun-11 freebsd-current list submittal.)

I reported more examples later in 2016 but
but have no records of replies, as  well as
Comment 2's 2017-Jan-25 submittal.

So while I think it was fixed, I do not have
a reference to just what specific attempted
fix finally got rid of the problem for cross
builds.