Problem found when building sysutils/e2fsprogs on FreeBSD 12.1-RELEASE-p1 powerpc 32 bit, running on Apple Powerbook 17". (e2fsprogs required to build sysutils/dsbmd) # cd /usr/ports/sysutils/e2fsprogs # make -DBATCH install clean ===> Building for e2fsprogs-1.45.3_2 ..... ..... gmake[1]: Entering directory '/usr/ports/sysutils/e2fsprogs/work/e2fsprogs-1.45.3/tests' Running e2fsprogs test suite... ..... f_messy_inode: bad file and directory acl pointers: ok f_inlinedata_dirblocks: check inline dir as two dirent blocks: failed i_qcow: create/convert raw/qcow2 images: ok ..... d_fallocate_bigalloc: fallocate sparse files and big files with bigalloc: ok 317 tests succeeded 1 tests failed Tests failed: f_inlinedata_dirblocks gmake[1]: *** [Makefile:396: test_post] Error 1 ..... ..... -Exit status is 0 +Entry '..' in Signal (11) SIGSEGV si_code=SEGV_MAPERR fault addr=0x2ff0 +Exit status is 8 *** Error code 1 NB: doc https://github.com/tytso/e2fsprogs/tree/master/tests mentions: "These images contain various forms of corrupted filesystem which e2fsck will correct. They are used as a regression test for e2fsck." (see full log attached)
Created attachment 210133 [details] Log build sysutils/e2fsprogs on powerpc
Ooops. That's bad. Unfortunately I don't have any powerpc stuff here, so I will be unable to reproduce things. I have updated the port to 1.45.4 yesterday, but am not sure if powerpc32 triggers CVE-2019-5094 here - see <http://e2fsprogs.sourceforge.net/e2fsprogs-release.html#1.45.4> I know that the self-tests of 1.45.4 passed for i386, amd64 on 11.3/12.0/12.1 and for 12.1 on MIPS64 and ARM64. So can you first retry with 1.45.4? It should have propagated to the mirrors now (for portsnap, or SVN) If that still fails, can you try recompiling with WITH_DEBUG=yes and then grab a backtrace from the program that caught SIGSEGV? We may need to send that upstream to Ted Y. Ts'o if 1.45.4 still crashes. Let me know if you need more detailed instructions.
Via portsnap, I've updated version to e2fsprogs-1.45.4 Error still present. So I run # make clean # make WITH_DEBUG=yes install clean (I guess it is the right command) Please find attached new log WITH_DEBUG
Created attachment 210171 [details] build log file WITH_DEBUG
Comment on attachment 210171 [details] build log file WITH_DEBUG it's this part we need debugged, from the f_inlinedata_dirblocks test: "SIGSEGV si_code=SEGV_MAPERR fault addr=0x2f6c" - can you get a stack backtrace of this crash, is there a core file you could get the backtrace from?
Hi Canardo, as written by Ted in a previous e-mail (invisible to Bugzilla), please see if there is a e2fsck.core file under the work directories' e2fsprogs*/tests directory (possibly: find . -name '*core') and if you don't know how to get a stack trace from it, upload the file and the e2fsck/e2fsck program for me somewhere. If there is a version that used to work, let us know which so we can see where between that version and v1.43.3 it broke.
Another approach would be adding USE_GCC=yes to the Makefile and see if a GCC-from-ports fares better than the stale GCC 4.2.1 from the base system, on powerpc.
Submitter wrote that he doesn't currently have access to his powerpc hardware and will see to it in 2020 when he regains access. Instructions for getting a backtrace: https://stackoverflow.com/questions/5745215/getting-stacktrace-from-core-dump possibly adapted if you build the port from source, for bash and similar shells: d=`make -V WRKSRC` gdb ${d}/e2fsck/e2fsck `find ${d} -name e2fsck.core` This works assuming that there is exactly one e2fsck.core file from the failing self-test. The compile should have been made WITH_DEBUG=yes (possibly re-build and re-install e2fsprogs-libuuid and e2fsprogs-libblkid also WITH_DEBUG=yes).
A commit references this bug: Author: mandree Date: Thu Jan 30 23:07:44 UTC 2020 New revision: 524661 URL: https://svnweb.freebsd.org/changeset/ports/524661 Log: sysutils/e2fsprogs: Try and enable backtrace() where possible. The configure script doesn't find backtrace() because it does not attempt linking against -lexecinfo -lelf. Note that currently backtrace() is broken at least on 12.1 amd64, and does not return any frames. Filed separately as <https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=243746> PR: 242798 (which prompts this change, but is not fixed) Changes: head/sysutils/e2fsprogs/Makefile
A commit references this bug: Author: mandree Date: Fri Jan 31 02:13:25 UTC 2020 New revision: 524662 URL: https://svnweb.freebsd.org/changeset/ports/524662 Log: Try to fix powerpc build, add LIBUNWIND option. We've had a USES+=compiler:c11 on powerpc64, see <https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=231570> extend and use it on all powerpc* in an attempt to fix PR 242798. If that doesn't help, add a LIBUNWIND option that makes e2fsck's signal catcher use libunwind instead of libexecinfo - FreeBSD's stack unwinder doesn't currently work from signal handlers (#243746), libunwind does and is supposed to work on powerpc and powerpc64, among others. PR: 242798 Reported by: canardo909@gmx.com Changes: head/sysutils/e2fsprogs/Makefile head/sysutils/e2fsprogs/files/extrapatch-e2fsck_sigcatcher.c
Looking at a powerpc64 issue that got solved per https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=231570 and in particular its attachment 197332 [details] here https://bz-attachments.freebsd.org/attachment.cgi?id=197332 ...this looks familiar. I've taken a shot in the dark with extending our "use newer compiler on powerpc64" choice to powerpc*. I have also added a LIBUNWIND option - best combined with WITH_DEBUG=yes to hopefully get a proper backtrace. @Canardo, can you please update your ports tree, confirm you have e2fsprogs 1.45.5_2 (PORTREVISION=2 in its makefile) and then rebuild that (after make clean)? If it still fails self-tests, please enable the LIBUNWIND option (make config), and re-build and re-install the port with added WITH_DEBUG=yes so we get the new backtrace from the log.
Uh, possibly libunwind doesn't build on powerpc - possibly it can be made to build by removing the ONLY_FOR_ARCH... line in its makefile, but that's an experiment with uncertain result.
(In reply to Matthias Andree from comment #11) Tests results are not OK /usr/ports/sysutils/e2fsprogs # more Makefile | grep "PORT" PORTNAME= e2fsprogs PORTVERSION= 1.45.5 PORTREVISION?= 2 ..... ..... ..... ===> Fetching all distfiles required by e2fsprogs-1.45.5_2 for building ===> Extracting for e2fsprogs-1.45.5_2 => SHA256 Checksum OK for e2fsprogs-1.45.5.tar.xz. ===> Patching for e2fsprogs-1.45.5_2 ===> Applying FreeBSD patches for e2fsprogs-1.45.5_2 ..... ..... ..... cd /usr/ports/sysutils/e2fsprogs/work/e2fsprogs-1.45.5/e2fsck && /bin/rm -f -f e2fsck && gmake e2fsck V=1 LIBS="../lib/libsupport.a ../lib/libext2fs.a ../lib/libcom_err.a /usr/lib/libexecinfo.a /usr/lib/libelf.a /usr/local/lib/libblkid.a /usr/local/lib/libuuid.a /usr/local/lib/libintl.a ../lib/libe2p.a" LIBMAGIC=/usr/lib/libmagic.a\ -lz gmake[2]: Entering directory '/usr/ports/sysutils/e2fsprogs/work/e2fsprogs-1.45.5/e2fsck' gcc9 -fstack-protector-strong -Wl,-rpath=/usr/local/lib/gcc9 -L/usr/local/lib/gcc9 -L/usr/local/lib -Wl,--rpath -Wl,/usr/local/lib/e2fsprogs -Wl,-rpath-link,../lib -rdynamic -o e2fsck unix.o e2fsck.o super.o pass1.o pass1b.o pass2.o pass3.o pass4.o pass5.o journal.o badblocks.o util.o dirinfo.o dx_dirinfo.o ehandler.o problem.o message.o quota.o recovery.o region.o revoke.o ea_refcount.o rehash.o logfile.o sigcatcher.o readahead.o extents.o ../lib/libsupport.a ../lib/libext2fs.a ../lib/libcom_err.a /usr/lib/libexecinfo.a /usr/lib/libelf.a /usr/local/lib/libblkid.a /usr/local/lib/libuuid.a /usr/local/lib/libintl.a ../lib/libe2p.a gmake[2]: Leaving directory '/usr/ports/sysutils/e2fsprogs/work/e2fsprogs-1.45.5/e2fsck' ===> checking that e2fsck depends on no shared objects outside /lib: FAIL ===> e2fsck depends on: /usr/local/lib/gcc9/libgcc_s.so.1 *** Error code 1 Stop. make[1]: stopped in /usr/ports/sysutils/e2fsprogs *** Error code 1 Then, I noticed I can't use libunwind /usr/ports/devel/libunwind # make fetch-recursive ===> Fetching all distfiles for libunwind-20170615 and dependencies ===> libunwind-20170615 is only for aarch64 amd64 i386, while you are running powerpc. *** Error code 1 Stop. make[1]: stopped in /usr/ports/devel/libunwind *** Error code 1 And regarding the "ONLY_FOR_ARCH" parameter, I don't find it in the Makefile /usr/ports/sysutils/e2fsprogs # more Makefile | grep -i "ARCH" . if ! empty(ARCH:Mpowerpc*) .if ${MASTERDIR} == ${.CURDIR} && ${ARCH} == "aarch64" .if !empty(PORT_OPTIONS:MNOTESTS) && (${OPSYS} == FreeBSD) && (${OSVERSION} >= 1200000 || ((${ARCH} != i386) && (${ARCH} != amd64)))
A commit references this bug: Author: mandree Date: Mon Feb 3 00:34:16 UTC 2020 New revision: 525006 URL: https://svnweb.freebsd.org/changeset/ports/525006 Log: Be more careful about what and how we link. Only link what we need (--as-needed) to get unnecessary dependencies out of the executables. Be sure to properly link libc dynamically and the rest statically, especially libgcc (-static-libgcc, should fix powerpc* builds => [1]) After two years, drop the "for now" LLD_UNSAFE=yes (r457508 by emaste@). As e2fsprogs dlopen()s libmagic these days, our LIBMAGIC override is useless. Instead patch libmagic.so.1 -> libmagic.so (12.1 calls it libmagic.so.4; patch to lib/support/plausible.c). Refresh files/patch-misc__Makefile.in PR: 242798 [1] Changes: head/sysutils/e2fsprogs/Makefile head/sysutils/e2fsprogs/files/patch-lib_support_plausible.c head/sysutils/e2fsprogs/files/patch-misc__Makefile.in
Canardo, could you try 1.45.5_3 (you'll need to "make config" and disable the LIBUNWIND option again)? It adds -static-libgcc, i. e. should link libgcc statically so the "check[...] that e2fsck depends on no shared objects outside /lib" should now pass.
(In reply to Matthias Andree from comment #15) Unfortunately test fails again with version 1.45.5_3 /usr/ports/sysutils/e2fsprogs # cat Makefile | grep "PORT" PORTNAME= e2fsprogs PORTVERSION= 1.45.5 PORTREVISION?= 3 # make config everything left by default, LIBUNWIND is not selected # make clean # make WITH_DEBUG=yes install clean ===> License GPLv2+ accepted by the user ===> e2fsprogs-1.45.5_3 depends on file: /usr/local/sbin/pkg - found ===> Fetching all distfiles required by e2fsprogs-1.45.5_3 for building ===> Extracting for e2fsprogs-1.45.5_3 ..... ..... 317 tests succeeded 1 tests failed Tests failed: f_inlinedata_dirblocks gmake[2]: *** [Makefile:395: test_post] Error 1 gmake[2]: Leaving directory '/usr/ports/sysutils/e2fsprogs/work/e2fsprogs-1.45.5/tests' --- f_inlinedata_dirblocks/expect.1 2020-01-06 23:10:17.000000000 +0000 +++ f_inlinedata_dirblocks.1.log 2020-02-04 22:25:24.795749000 +0000 @@ -1,25 +1,4 @@ Pass 1: Checking inodes, blocks, and sizes Pass 2: Checking directory structure -Entry '..' in ??? (12) has invalid inode #: 1752440867. -Clear? yes ..... ..... -test_filesys: ***** FILE SYSTEM WAS MODIFIED ***** -test_filesys: 12/128 files (0.0% non-contiguous), 17/512 blocks -Exit status is 1 +Entry '..' in Signal (11) SIGSEGV si_code=SEGV_MAPERR fault addr=0x2ed8 +Exit status is 8 --- f_inlinedata_dirblocks/expect.2 2020-01-06 23:10:17.000000000 +0000 +++ f_inlinedata_dirblocks.2.log 2020-02-04 22:25:24.842697000 +0000 @@ -1,7 +1,4 @@ Pass 1: Checking inodes, blocks, and sizes Pass 2: Checking directory structure -Pass 3: Checking directory connectivity -Pass 4: Checking reference counts -Pass 5: Checking group summary information -test_filesys: 12/128 files (0.0% non-contiguous), 17/512 blocks -Exit status is 0 +Entry '..' in Signal (11) SIGSEGV si_code=SEGV_MAPERR fault addr=0x2ed8 +Exit status is 8 *** Error code 1 Stop. make[1]: stopped in /usr/ports/sysutils/e2fsprogs *** Error code 1 See full log WITH_DEBUG=yes attached
Created attachment 211361 [details] FreeBSD.bug.242798.e2fsprogs_v1.45.5_3-FULL.LOG.WITH.DEBUG.txt.zip
:-( Then we'll need to do some more debugging along these lines: cd /usr/ports/sysutils/e2fsprogs make clean NOCLEANDEPENDS=1 make WITH_DEBUG=yes configure # this stops before building so we can edit source code edit work/e2fsprogs-1.45.5/e2fsck/sigcatcher.c just above line 400, erase the entire line that reads "sigaction(SIGSEGV, &sa, 0);", this makes sure that the FreeBSD kernel with drop the core on SIGSEGV. make WITH_DEBUG=yes # this completes the build => now you should get a core file from the tests, work/e2fsprogs-1.45.5/tests/e2fsck.core gdb work/e2fsprogs-1.45.5/e2fsck/e2fsck work/e2fsprogs-1.45.5/tests/e2fsck.core inside gdb, optionally: set pagination off (see how to disable pagination prompt in GDB) inside gdb: bt full inside gdb: bt
(In reply to Matthias Andree from comment #18) Core file generated successfully ! Please find attach the results of bt* commands Full log and .core available if needed
Created attachment 211391 [details] FreeBSD.bug.242798.e2fsprogs_v1.45.5_3-STACKTRACE
A commit references this bug: Author: mandree Date: Mon Feb 17 21:54:24 UTC 2020 New revision: 526434 URL: https://svnweb.freebsd.org/changeset/ports/526434 Log: sysutils/e2fsprogs: Fix powerpc32 build PR: 242798 Reported by: canardo909@gmx.com Obtained from: Theodore Y. Ts'o <tytso@mit.edu> (upstream maintainer) MFH: 2020Q1 (fixes build on powerpc) Changes: head/sysutils/e2fsprogs/Makefile head/sysutils/e2fsprogs/files/patch-lib_ext2fs_swapfs.c
Fixed on head, MFH request sent to ports-secteam@ (automatically from the commit) - will invoke blanket approval.
A commit references this bug: Author: mandree Date: Mon Feb 17 22:14:33 UTC 2020 New revision: 526435 URL: https://svnweb.freebsd.org/changeset/ports/526435 Log: MFH: r526434 sysutils/e2fsprogs: Fix powerpc32 build/self-tests libext2fs: avoid array buffer overruns caused by insane directory blocks PR: 242798 Reported by: canardo909@gmx.com Obtained from: Theodore Y. Ts'o <tytso@mit.edu> (upstream maintainer) Approved by: ports-secteam (blanket, fixing broken self-test by small patch) Changes: _U branches/2020Q1/ branches/2020Q1/sysutils/e2fsprogs/Makefile branches/2020Q1/sysutils/e2fsprogs/files/patch-lib_ext2fs_swapfs.c