Bug 247020 - [panic][lixux]: kernel panic in linux_thread_detach at /usr/src/sys/compat/linux/linux_fork.c:462
Summary: [panic][lixux]: kernel panic in linux_thread_detach at /usr/src/sys/compat/li...
Status: Closed FIXED
Alias: None
Product: Base System
Classification: Unclassified
Component: kern (show other bugs)
Version: 12.1-RELEASE
Hardware: amd64 Any
: --- Affects Only Me
Assignee: Konstantin Belousov
URL:
Keywords: panic
Depends on:
Blocks:
 
Reported: 2020-06-05 21:11 UTC by Martin Filla
Modified: 2020-07-20 14:23 UTC (History)
5 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Martin Filla 2020-06-05 21:11:24 UTC
Hello,
when trying to execute linux binary with gdb debug then have
kernel panic immediately.

kldload linux64
gdb ./linux_binary
(gdb) run

Unread portion of the kernel message buffer:
panic: thread_detach: emuldata not found.

cpuid = 6
time = 1591381865
KDB: stack backtrace:
db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe0075872930
vpanic() at vpanic+0x19d/frame 0xfffffe0075872980
panic() at panic+0x43/frame 0xfffffe00758729e0
linux_thread_detach() at linux_thread_detach+0x9e/frame 0xfffffe0075872a30
thread_suspend_check() at thread_suspend_check+0x318/frame 0xfffffe0075872a70
ast() at ast+0x544/frame 0xfffffe0075872ab0
doreti_ast() at doreti_ast+0x1f/frame 0x7fffdfffdd90
Uptime: 4m46s
Dumping 814 out of 8042 MB:..2%..12%..22%..32%..42%..52%..61%..71%..81%..91%

__curthread () at /usr/src/sys/amd64/include/pcpu.h:234
234		__asm("movq %%gs:%P1,%0" : "=r" (td) : "n" (OFFSETOF_CURTHREAD));
(kgdb) bt
#0  __curthread () at /usr/src/sys/amd64/include/pcpu.h:234
#1  doadump (textdump=1) at /usr/src/sys/kern/kern_shutdown.c:371
#2  0xffffffff80bb70b0 in kern_reboot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:451
#3  0xffffffff80bb74f9 in vpanic (fmt=<optimized out>, ap=<optimized out>) at /usr/src/sys/kern/kern_shutdown.c:877
#4  0xffffffff80bb7263 in panic (fmt=<unavailable>) at /usr/src/sys/kern/kern_shutdown.c:804
#5  0xffffffff82f4e1be in linux_thread_detach (td=0xfffff8017a48c5e0) at /usr/src/sys/compat/linux/linux_fork.c:462
#6  0xffffffff80bcbfa8 in thread_suspend_check (return_instead=0) at /usr/src/sys/kern/kern_thread.c:1010
#7  0xffffffff80c167c4 in ast (framep=0xfffffe0075872ac0) at /usr/src/sys/kern/subr_trap.c:342
#8  0xffffffff810670f9 in doreti_ast () at /usr/src/sys/amd64/amd64/exception.S:1149
#9  0x0000000800bb7008 in ?? ()
#10 0x000000000000000f in ?? ()
#11 0x0000000000000000 in ?? ()
(kgdb) list *0xffffffff82f4e1be
0xffffffff82f4e1be is at /usr/src/sys/compat/linux/linux_fork.c:462.
457		struct linux_emuldata *em;
458		int *child_clear_tid;
459		int error;
460	
461		em = em_find(td);
462		KASSERT(em != NULL, ("thread_detach: emuldata not found.\n"));
463	
464		LINUX_CTR1(thread_detach, "thread(%d)", em->em_tid);
465	
466		release_futexes(td, em);
(kgdb) 
467	
468		child_clear_tid = em->child_clear_tid;
469	
470		if (child_clear_tid != NULL) {
471	
472			LINUX_CTR2(thread_detach, "thread(%d) %p",
473			    em->em_tid, child_clear_tid);
474	
475			error = suword32(child_clear_tid, 0);
476			if (error != 0)
(kgdb) list 
477				return;
478	
479			cup.uaddr = child_clear_tid;
480			cup.op = LINUX_FUTEX_WAKE;
481			cup.val = 1;		/* wake one */
482			cup.timeout = NULL;
483			cup.uaddr2 = NULL;
484			cup.val3 = 0;
485			error = linux_sys_futex(td, &cup);
486			/*
Comment 1 Martin Filla 2020-06-06 08:32:19 UTC
mount |grep /compat/linux
devfs on /compat/linux/dev (devfs, local, multilabel)
linprocfs on /compat/linux/proc (linprocfs, local)
linsysfs on /compat/linux/sys (linsysfs, local)


freebsd-version -ku 
12.1-RELEASE-p5
12.1-RELEASE-p5


pkg info | grep "linux"
drm-legacy-kmod-g20200306      Legacy DRM modules for the linuxkpi-based KMS components
gpu-firmware-kmod-g20200130    Firmware modules for the linuxkpi-based KMS components
linux-c7-alsa-lib-1.1.8        Advanced Linux Sound Architecture libraries (Linux CentOS 7.7.1908)
linux-c7-alsa-plugins-oss-1.1.6_3 OSS plugin for ALSA (Linux CentOS 7.7.1908)
linux-c7-alsa-plugins-pulseaudio-1.1.6_1 PulseAudio plugin for ALSA (Linux CentOS 7.7.1908)
linux-c7-alsa-utils-1.1.8      Advanced Linux Sound Architecture utilities (Linux CentOS 7.7.1908)
linux-c7-aspell-0.60.6.1_1     Spelling checker with better logic than ispell (Linux CentOS 7.7.1908)
linux-c7-atk-2.28.1            Accessibility Toolkit (Linux CentOS 7.7.1908)
linux-c7-audiofile-0.3.6_1     Implementation of the Audio File Library (Linux CentOS 7.7.1908)
linux-c7-avahi-libs-0.6.31_3   Libraries for Avahi (Linux CentOS 7.7.1908)
linux-c7-cairo-1.15.12_1       Vector graphics library Cairo (Linux CentOS 7.7.1908)
linux-c7-cups-libs-1.6.3_7     Common UNIX Printing System libraries
linux-c7-curl-7.29.0_9         Tool for transferring files with URL syntax (Linux CentOS 7.7.1908)
linux-c7-cyrus-sasl-lib-2.1.26_4 RFC 2222 SASL (Simple Authentication and Security Layer) (Linux CentOS 7.7.1908)
linux-c7-dbus-glib-0.100_1     GLib bindings for D-Bus (Linux CentOS 7.7.1908)
linux-c7-dbus-libs-1.10.24_1   Libraries for accessing D-BUS (Linux CentOS 7.7.1908)
linux-c7-devtools-7.7.1908_6   Linux compilers and base libraries (CentOS 7.7.1908)
linux-c7-dri-18.3.4_1          Mesa libGL runtime libraries (Linux CentOS 7.7.1908)
linux-c7-elfutils-libelf-0.176 ELF file handling library (CentOS 7.7.1908)
linux-c7-elfutils-libs-0.176   ELF file handling libraries (CentOS 7.7.1908)
linux-c7-expat-2.1.0_2         XML 1.0 parser written in C (Linux CentOS 7.7.1908)
linux-c7-flac-libs-1.3.0_2     Free lossless audio codec (Linux CentOS 7.7.1908)
linux-c7-fontconfig-2.13.0     XML-based font configuration API for X Windows (Linux CentOS 7.7.1908)
linux-c7-freetype-2.8_1        Font rendering engine (Linux CentOS 7.7.1908)
linux-c7-fribidi-1.0.2_1       Unicode Bidirectional Algorithm (Linux CentOS 7.7.1908)
linux-c7-gdk-pixbuf2-2.36.12   Graphic library for GTK+ (Linux CentOS 7.7.1908)
linux-c7-glx-utils-8.3.0_1     Mesa GLX utilities (Linux CentOS 7.7.1908)
linux-c7-gnutls-3.3.29         GNU Transport Layer Security library (Linux CentOS 7.7.1908)
linux-c7-graphite2-1.3.10      Rendering capabilities for complex non-Roman writing systems (Linux CentOS 7.7.1908)
linux-c7-harfbuzz-1.7.5        OpenType text shaping engine (Linux CentOS 7.7.1908)
linux-c7-jasper-libs-1.900.1_5 JPEG-2000 reference implementation (Linux CentOS 7.7.1908)
linux-c7-jbigkit-libs-2.0_2    Lossless compression for bi-level images (Linux CentOS 7.7.1908)
linux-c7-jpeg-1.2.90_4         SIMD-accelerated JPEG codec (Linux CentOS 7.7.1908)
linux-c7-libasyncns-0.8_1      Library for executing DNS queries (Linux CentOS 7.7.1908)
linux-c7-libdrm-2.4.91         Interface to kernel Direct Rendering Module (Linux CentOS 7.6.1810)
linux-c7-libgcrypt-1.5.3_4     Crypto library based on code used in GnuPG (Linux CentOS 7.7.1908)
linux-c7-libgfortran-4.8.5_5   Runtime libraries for gfortran (Linux Centos 7.7.1908)
linux-c7-libglade2-2.6.4_1     GNOME glade library (Linux CentOS 7.7.1908)
linux-c7-libglvnd-1.0.1        GL Vendor-Neutral Dispatch library (Linux CentOS 7.7.1908)
linux-c7-libgpg-error-1.12_2   Common error values for all GnuPG components (Linux CentOS 7.7.1908)
linux-c7-libogg-1.3.0_1        Ogg bitstream library (Linux CentOS 7.7.1908)
linux-c7-libpciaccess-0.14     Generic PCI access library (CentOS 7.7.1908)
linux-c7-libpng-1.5.13_2       Library for manipulating PNG images (Linux CentOS 7.7.1908)
linux-c7-libsigc++20-2.10.0    Callback Framework for C++ (Linux CentOS 7.7.1908)
linux-c7-libsndfile-1.0.25_3   Reading and writing files containing sampled sound (like WAV or AIFF) (Linux CentOS 7.7.1908)
linux-c7-libssh2-1.8.0         Library implementing the SSH2 protocol (Linux CentOS 7.7.1908)
linux-c7-libtasn1-4.10         ASN.1 structure parser library (Linux CentOS 7.7.1908)
linux-c7-libthai-0.1.14_1      Thai language support library (Linux CentOS 7.7.1908)
linux-c7-libtheora-1.1.1_1     Theora Video Compression Codec (Linux CentOS 7.7.1908)
linux-c7-libtiff-4.0.3_4       Library routines for working with TIFF images (Linux CentOS 7.7.1908)
linux-c7-libv4l-0.9.5_2        Video4Linux library (Linux CentOS 7.7.1908)
linux-c7-libvorbis-1.3.3_2     Audio compression codec library (Linux CentOS 7.7.1908)
linux-c7-libxml2-2.9.1_1       Library providing XML and HTML support (Linux CentOS 7.7.1908)
linux-c7-lz4-1.7.5_1           LZ4 compression library and utilities (Linux CentOS 7.7.1908)
linux-c7-motif-2.3.4_6         Motif toolkit libraries (Linux CentOS 7.7.1908)
linux-c7-nettle-2.7.1          Low-level cryptographic library (Linux CentOS 7.7.1908)
linux-c7-nspr-4.21.0           Netscape Portable Runtime (Linux CentOS 7.7.1908)
linux-c7-nss-3.44.0_1          Network Security Services (Linux CentOS 7.7.1908)
linux-c7-openal-soft-1.16.0_3  3D positional spatialized sound library (Linux CentOS 7.7.1908)
linux-c7-openldap-2.4.44_1     LDAP libraries (Linux CentOS 7.7.1908)
linux-c7-p11-kit-0.23.5        Library for loading and enumerating of PKCS#11 modules (Linux CentOS 7.7.1908)
linux-c7-pango-1.42.4_1        Pango library (Linux CentOS 7.7.1908)
linux-c7-pixman-0.34.0         Low-level pixel manipulation library (Linux CentOS 7.7.1908)
linux-c7-pulseaudio-libs-10.0_2 Libraries for PulseAudio clients (Linux CentOS 7.7.1908)
linux-c7-pulseaudio-utils-10.0_1 Utils for PulseAudio clients (Linux CentOS 7.7.1908)
linux-c7-qt-4.8.7_2            RPM of QT4 (Linux CentOS 7.7.1908)
linux-c7-qt-x11-4.8.7_2        Cross-platform application and UI framework (Linux CentOS 7.7.1908)
linux-c7-qtwebkit-2.3.4_2      Qt WebKit implementation (Linux CentOS 7.7.1908)
linux-c7-sdl-1.2.15_2          Cross-platform multi-media development API (Linux CentOS 7.7.1908)
linux-c7-sdl20-2.0.10          Cross-platform multi-media development API (Linux CentOS 7.7.1908)
linux-c7-sdl_image-1.2.12_2    Simple library to load images as SDL interfaces (Linux CentOS 7.7.1908)
linux-c7-sdl_mixer-1.2.12_1    Sample multi-channel audio mixer library (Linux CentOS 7.7.1908)
linux-c7-sdl_ttf-2.0.11_2      Simple library to load True Type Fonts as SDL interfaces (Linux CentOS 7.7.1908)
linux-c7-sqlite-3.7.17_2       Library that implements an embeddable SQL database engine (Linux CentOS 7.7.1908)
linux-c7-systemd-libs-219_5    Libraries for accessing systemd (Linux CentOS 7.7.1908)
linux-c7-tcl85-8.5.13_1        Tool Command Language (Linux CentOS 7.7.1908)
linux-c7-tcp_wrappers-libs-7.6_2 Libraries tcp wrappers (Linux CentOS 7.7.1908)
linux-c7-tk85-8.5.13_1         Graphical toolkit for TCL (Linux CentOS 7.7.1908)
linux-c7-trousers-0.3.14_1     Open-source TCG Software Stack (Linux CentOS 7.7.1908)
linux-c7-wayland-1.15.0        Wayland compositor libraries (Linux CentOS 7.7.1908)
linux-c7-xorg-libs-7.7_8       Xorg libraries (Linux CentOS 7.7.1908)
linux-doom3-demo-1.1.1286_4    DOOM III demo for Linux
linux_base-c7-7.7.1908_1       Base set of packages needed in Linux mode (Linux CentOS 7.7.1908)
linuxlibertine-g-20120116_2    Linux Libertine G and Linux Biolinum G fonts
syslinux-6.03                  Boot loader for the Linux operating system
Comment 2 Konstantin Belousov freebsd_committer 2020-06-07 16:52:33 UTC
Can you provide statically linked linux binary that can be used to demonstrate the issue ?
Comment 3 Martin Filla 2020-06-08 10:28:48 UTC
(In reply to Konstantin Belousov from comment #2)

yes, for example app discord or eclipse for linux..
Comment 4 Alex S 2020-06-10 13:22:06 UTC
(In reply to Martin FIlla from comment #3)

> yes, for example app discord or eclipse for linux..

If possible, a direct link to a smaller program would be nice. Or maybe this is reproducible with one of the linux-* packages?
Comment 5 Martin Filla 2020-06-10 14:41:36 UTC
(In reply to Alex S from comment #4)

I have installed linux_base-c7-7.7.1908_1 package.
I have problem with almost everyone program for linux.
Comment 6 Alex S 2020-06-10 14:57:19 UTC
(In reply to Martin FIlla from comment #5)

Here is how it looks for me on FreeBSD 12.1-RELEASE (with yet another Electron application):

% gdb ./teams-insiders 
GNU gdb (GDB) 9.1 [GDB v9.1 for FreeBSD]
Copyright (C) 2020 Free Software Foundation, Inc.
<...>
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from ./teams-insiders...
(No debugging symbols found in ./teams-insiders)
(gdb) run
Starting program: /usr/home/<...>/teams/usr/share/teams-insiders/teams-insiders 
[New LWP 101520 of process 62832]

Thread 1 "teams-insiders" received signal SIGTRAP, Trace/breakpoint trap.
0x0000000005ac6f01 in ?? ()

No panic here, obviously.
Comment 7 Martin Filla 2020-06-10 16:07:51 UTC
(In reply to Alex S from comment #6)

pid 74578 (teams), jid 0, uid 1001: exited on signal 5 (core dumped)
pid 81034 (teams), jid 0, uid 1001: exited on signal 5 (core dumped)
pid 3688 (teams), jid 0, uid 1001: exited on signal 5 (core dumped)
pid 7404 (teams), jid 0, uid 1001: exited on signal 5 (core dumped)


Fatal trap 12: page fault while in kernel mode
cpuid = 7; apic id = 07
fault virtual address	= 0x18
fault code		= supervisor read data, page not present
instruction pointer	= 0x20:0xffffffff82f5b682
stack pointer	        = 0x28:0xfffffe0071908980
frame pointer	        = 0x28:0xfffffe00719089e0
code segment		= base rx0, limit 0xfffff, type 0x1b
			= DPL 0, pres 1, long 1, def32 0, gran 1
processor eflags	= interrupt enabled, resume, IOPL = 0
current process		= 13595 (fish)
trap number		= 12
panic: page fault
cpuid = 7
time = 1591804935
KDB: stack backtrace:
#0 0xffffffff80c1d307 at kdb_backtrace+0x67
#1 0xffffffff80bd063d at vpanic+0x19d
#2 0xffffffff80bd0493 at panic+0x43
#3 0xffffffff810a7dcc at trap_fatal+0x39c
#4 0xffffffff810a7e19 at trap_pfault+0x49
#5 0xffffffff810a740f at trap+0x29f
#6 0xffffffff81081bec at calltrap+0x8
#7 0xffffffff82f503d1 at linux_thread_detach+0x21
#8 0xffffffff80be5adf at thread_suspend_check+0x41f
#9 0xffffffff80c32ee9 at ast+0x3b9
#10 0xffffffff810850f9 at doreti_ast+0x1f
Comment 8 Konstantin Belousov freebsd_committer 2020-06-16 21:05:56 UTC
https://reviews.freebsd.org/D25293
Try this
Comment 9 commit-hook freebsd_committer 2020-06-18 20:50:54 UTC
A commit references this bug:

Author: kib
Date: Thu Jun 18 20:49:57 UTC 2020
New revision: 362342
URL: https://svnweb.freebsd.org/changeset/base/362342

Log:
  Fix execution of linux binary from multithreaded non-Linux process.

  If multithreaded non-Linux process execs Linux binary, then non-Linux
  threads different from the one that execing are cleared by
  single-threading at boundary, and then terminating them in
  post_execve(). Since at that time the process is already switched to
  linux ABI, linuxolator is involved in the thread clearing on boundary,
  but cannot find the emul data.

  Handle it by pre-creating emuldata for all threads in the execing process.

  Also remove a code in linux_proc_exec() handler that cleared emul data
  for other threads when execing from multithreaded Linux process. It is
  excessive.

  PR:	247020
  Reported by:	Martin FIlla <freebsd@sysctl.cz>
  Reported by:	Henrique L. Amorim, Independent Security Researcher
  Reported by:	Rodrigo Rubira Branco (BSDaemon), Amazon Web Services
  Reviewed by:	markj
  Tested by:	trasz
  Sponsored by:	The FreeBSD Foundation
  MFC after:	1 week
  Differential revision:	https://reviews.freebsd.org/D25293

Changes:
  head/sys/compat/linux/linux_emul.c
Comment 10 Mark Johnston freebsd_committer 2020-07-20 14:23:29 UTC
Merged to stable/12 in r362605.
Merged to stable/11 in r362606.