Created attachment 223747 [details]
pstree output when 0% CPU hang occurs

Introduction: In this issue, lang/rust shall be compiled for armv6; the exact same environment is used both with an emulation provided by qemu-user-static and natively; when using the emulation the build process eventually hangs with 0% CPU, whereas natively it always proceeds. Emulation is desirable because it is much faster than the armv6 machine.

Scenario:
- FreeBSD main at 51dfae383bf6298af9e6d816a78b92b6f34d68be (used in all installations in the following lines, regardless of armv6 or amd64)
- iSCSI-exported disk ("disk925") carrying a complete FreeBSD armv6 installation
- Raspberry Pi B+ ("rpi-b") carrying a complete FreeBSD armv6 installation (built using "make release")
- bhyve amd64 virtual machine ("v903")
- The goal is to update the armv6 ports using portmaster; amongst them, lang/rust needs to be built
- Running v903 as a bhyve client under FreeBSD 12.2
- In v903, emulators/qemu-user-static is available, and /etc/rc.conf contains "qemu_user_static_enable="YES""
- Attaching disk925 in v903; mounting it in /d/925s2a, mounting the special filesystems underneath (/dev, /dev/fd, /proc, /tmp)
- /usr/ports is NFS-exported from another host and mounted in v903 both in /net/... and in /d/925s2a/net/... using autofs  (most likely not important to this issue)
- In v903, issuing "chroot /d/925s2a"
- This opens a qemu-emulated zsh
- Issue "cd /usr/ports/lang/rust"
- Issue "TMPDIR=/usr/tmp make" (the standard TMPDIR=/tmp is too small; most likely not relevant to this issue)

Result:
- After compiling for an extended period of time progress stops and the qemu processes just sit there idle.
- It seems that this mostly happens at the linking stage of rust libraries.
- Example: Here is the currently visible output, with 0% CPU load:

   Compiling rustc-main v0.0.0 (/usr/tmp/net/hal/z/SRC/FreeBSD/ports/head/lang/rust/work/rustc-1.51.0-src/compiler/rustc)
     Running `/usr/tmp/net/hal/z/SRC/FreeBSD/ports/head/lang/rust/work/rustc-1.51.0-src/build/bootstrap/debug/rustc --crate-name rustc_main --edition=2018 compiler/rustc/src/main.rs --error-format=json --json=diagnostic-rendered-ansi --crate-type bin --emit=dep-info,link -C opt-level=3 -C embed-bitcode=no -C debuginfo=0 --cfg 'feature="llvm"' --cfg 'feature="max_level_info"' -C metadata=33c78eec63c05f4e -C extra-filename=-33c78eec63c05f4e --out-dir /usr/tmp/net/hal/z/SRC/FreeBSD/ports/head/lang/rust/work/rustc-1.51.0-src/build/armv6-unknown-freebsd/stage1-rustc/armv6-unknown-freebsd/release/deps --target armv6-unknown-freebsd -C linker=/usr/tmp/net/hal/z/SRC/FreeBSD/ports/head/lang/rust/work/cc-wrapper -L dependency=/usr/tmp/net/hal/z/SRC/FreeBSD/ports/head/lang/rust/work/rustc-1.51.0-src/build/armv6-unknown-freebsd/stage1-rustc/armv6-unknown-freebsd/release/deps -L dependency=/usr/tmp/net/hal/z/SRC/FreeBSD/ports/head/lang/rust/work/rustc-1.51.0-src/build/armv6-unknown-freebsd/stage1-rustc/release/deps --extern rustc_codegen_ssa=/usr/tmp/net/hal/z/SRC/FreeBSD/ports/head/lang/rust/work/rustc-1.51.0-src/build/armv6-unknown-freebsd/stage1-rustc/armv6-unknown-freebsd/release/deps/librustc_codegen_ssa-0891b87bc3d4da8f.rlib --extern rustc_driver=/usr/tmp/net/hal/z/SRC/FreeBSD/ports/head/lang/rust/work/rustc-1.51.0-src/build/armv6-unknown-freebsd/stage1-rustc/armv6-unknown-freebsd/release/deps/librustc_driver-623729bd2d57ce51.so -Zmacro-backtrace '-Clink-args=-Wl,-rpath,$ORIGIN/../lib' -Ztls-model=initial-exec -Zunstable-options '-Wrustc::internal' -Cprefer-dynamic -Z binary-dep-depinfo -L native=/usr/tmp/net/hal/z/SRC/FreeBSD/ports/head/lang/rust/work/rustc-1.51.0-src/build/armv6-unknown-freebsd/stage1-rustc/armv6-unknown-freebsd/release/build/psm-fdd3708a2d632057/out -L native=/usr/tmp/net/hal/z/SRC/FreeBSD/ports/head/lang/rust/work/rustc-1.51.0-src/build/armv6-unknown-freebsd/stage1-rustc/armv6-unknown-freebsd/release/build/rustc_llvm-6f28977a172d46d4/out -L native=/usr/tmp/net/hal/z/SRC/FreeBSD/ports/head/lang/rust/work/rustc-1.51.0-src/build/armv6-unknown-freebsd/llvm/build/lib`
rustc command: "LD_LIBRARY_PATH"="/usr/tmp/net/hal/z/SRC/FreeBSD/ports/head/lang/rust/work/rustc-1.51.0-src/build/armv6-unknown-freebsd/stage1/lib:/usr/tmp/net/hal/z/SRC/FreeBSD/ports/head/lang/rust/work/rustc-1.51.0-src/build/armv6-unknown-freebsd/stage1-rustc/release/deps:/usr/tmp/net/hal/z/SRC/FreeBSD/ports/head/lang/rust/work/rustc-1.51.0-src/build/armv6-unknown-freebsd/stage1/lib" "/usr/tmp/net/hal/z/SRC/FreeBSD/ports/head/lang/rust/work/rustc-1.51.0-src/build/armv6-unknown-freebsd/stage1/bin/rustc" "--crate-name" "rustc_main" "--edition=2018" "compiler/rustc/src/main.rs" "--error-format=json" "--json=diagnostic-rendered-ansi" "--crate-type" "bin" "--emit=dep-info,link" "-C" "opt-level=3" "-C" "embed-bitcode=no" "-C" "debuginfo=0" "--cfg" "feature=\"llvm\"" "--cfg" "feature=\"max_level_info\"" "-C" "metadata=33c78eec63c05f4e" "-C" "extra-filename=-33c78eec63c05f4e" "--out-dir" "/usr/tmp/net/hal/z/SRC/FreeBSD/ports/head/lang/rust/work/rustc-1.51.0-src/build/armv6-unknown-freebsd/stage1-rustc/armv6-unknown-freebsd/release/deps" "--target" "armv6-unknown-freebsd" "-C" "linker=/usr/tmp/net/hal/z/SRC/FreeBSD/ports/head/lang/rust/work/cc-wrapper" "-L" "dependency=/usr/tmp/net/hal/z/SRC/FreeBSD/ports/head/lang/rust/work/rustc-1.51.0-src/build/armv6-unknown-freebsd/stage1-rustc/armv6-unknown-freebsd/release/deps" "-L" "dependency=/usr/tmp/net/hal/z/SRC/FreeBSD/ports/head/lang/rust/work/rustc-1.51.0-src/build/armv6-unknown-freebsd/stage1-rustc/release/deps" "--extern" "rustc_codegen_ssa=/usr/tmp/net/hal/z/SRC/FreeBSD/ports/head/lang/rust/work/rustc-1.51.0-src/build/armv6-unknown-freebsd/stage1-rustc/armv6-unknown-freebsd/release/deps/librustc_codegen_ssa-0891b87bc3d4da8f.rlib" "--extern" "rustc_driver=/usr/tmp/net/hal/z/SRC/FreeBSD/ports/head/lang/rust/work/rustc-1.51.0-src/build/armv6-unknown-freebsd/stage1-rustc/armv6-unknown-freebsd/release/deps/librustc_driver-623729bd2d57ce51.so" "-Zmacro-backtrace" "-Clink-args=-Wl,-rpath,$ORIGIN/../lib" "-Ztls-model=initial-exec" "-Zunstable-options" "-Wrustc::internal" "-Cprefer-dynamic" "-Z" "binary-dep-depinfo" "-L" "native=/usr/tmp/net/hal/z/SRC/FreeBSD/ports/head/lang/rust/work/rustc-1.51.0-src/build/armv6-unknown-freebsd/stage1-rustc/armv6-unknown-freebsd/release/build/psm-fdd3708a2d632057/out" "-L" "native=/usr/tmp/net/hal/z/SRC/FreeBSD/ports/head/lang/rust/work/rustc-1.51.0-src/build/armv6-unknown-freebsd/stage1-rustc/armv6-unknown-freebsd/release/build/rustc_llvm-6f28977a172d46d4/out" "-L" "native=/usr/tmp/net/hal/z/SRC/FreeBSD/ports/head/lang/rust/work/rustc-1.51.0-src/build/armv6-unknown-freebsd/llvm/build/lib" "-Wrust_2018_idioms" "-Wunused_lifetimes" "-Wsemicolon_in_expressions_from_macros" "--sysroot" "/usr/tmp/net/hal/z/SRC/FreeBSD/ports/head/lang/rust/work/rustc-1.51.0-src/build/armv6-unknown-freebsd/stage1" "-Z" "force-unstable-if-unmarked"
sysroot: "/usr/tmp/net/hal/z/SRC/FreeBSD/ports/head/lang/rust/work/rustc-1.51.0-src/build/armv6-unknown-freebsd/stage1"
libdir: "/usr/tmp/net/hal/z/SRC/FreeBSD/ports/head/lang/rust/work/rustc-1.51.0-src/build/armv6-unknown-freebsd/stage1/lib"
    Building [=======================> ] 215/216: rustc-main(bin)

Attached is the associated pstree.

Scenario (continued; the purpose is to continue the compilation on the native armv6 machine):
- Issue ^C
- Exit the emulated zsh (^D)
- Unmount /d/925s2a etc. from v903
- Detach disk925 from v903
- Attach disk925 in rpi-b
- Mount /d/925s2a as well es /d/925s2a/dev etc. as well as the automounted directories (/d/925s2a/net; to reach /usr/ports) in rpi-b
- Issue "chroot /d/925s2a" (starts a zsh, this time running natively in exactly the same environment previously used with qemu-user-static on v903)
- Issue "cd /usr/ports/lang/rust"
- Issue "TMPDIR=/usr/tmp make"

Result (continued):
- The compilation restarts and proceeds alright (albeit much slower than with the qemu emulation)

Scenario (continued):
- After successfully passing the offending compile (or link?) on the RPI-B+, again issue ^C, tear down everyting from the RPI-B+, reattach everything again in the v903 machine, and continue with building the port until the next hang occurs.

Note:
- This has happened for a very long time and with several updates to qemu-user-static.

-- Martin