Bug 236383 - [ral] Kernel panic in ral(4)
Summary: [ral] Kernel panic in ral(4)
Status: Open
Alias: None
Product: Base System
Classification: Unclassified
Component: kern (show other bugs)
Version: 12.0-STABLE
Hardware: amd64 Any
: --- Affects Only Me
Assignee: freebsd-net mailing list
URL:
Keywords: panic, patch
Depends on:
Blocks:
 
Reported: 2019-03-08 07:40 UTC by Sergey Anokhin
Modified: 2019-03-19 14:35 UTC (History)
1 user (show)

See Also:


Attachments
Add wlock for priv modification to ng_iface_shutdown() (996 bytes, patch)
2019-03-16 19:21 UTC, Eugene Grosbein
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Sergey Anokhin 2019-03-08 07:40:14 UTC
Hi All,

12.0-STABLE FreeBSD 12.0-STABLE #2 r343904M

Strange panic, me don't know how to reproduce:

# kgdb /boot/kernel/kernel /var/crash/vmcore.last
GNU gdb (GDB) 8.2.1 [GDB v8.2.1 for FreeBSD]
Copyright (C) 2018 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-portbld-freebsd12.0".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /boot/kernel/kernel...Reading symbols from /usr/lib/debug//boot/kernel/kernel.debug...done.
done.

Unread portion of the kernel message buffer:


Fatal trap 12: page fault while in kernel mode
cpuid = 1; apic id = 01
fault virtual address   = 0xb10
fault code              = supervisor read data, page not present
instruction pointer     = 0x20:0xffffffff809019d2
stack pointer           = 0x28:0xfffffe0000464a70
frame pointer           = 0x28:0xfffffe0000464aa0
code segment            = base rx0, limit 0xfffff, type 0x1b
                        = DPL 0, pres 1, long 1, def32 0, gran 1
processor eflags        = interrupt enabled, resume, IOPL = 0
current process         = 12 (irq22: ral0)
trap number             = 12
panic: page fault
cpuid = 1
time = 1552029335
KDB: stack backtrace:
#0 0xffffffff80c531c7 at kdb_backtrace+0x67
#1 0xffffffff80c07143 at vpanic+0x1a3
#2 0xffffffff80c06f93 at panic+0x43
#3 0xffffffff8118d9ff at trap_fatal+0x35f
#4 0xffffffff8118da59 at trap_pfault+0x49
#5 0xffffffff8118d07e at trap+0x29e
#6 0xffffffff81168af5 at calltrap+0x8
#7 0xffffffff80901693 at rt2860_intr+0x803
#8 0xffffffff80bca024 at ithread_loop+0x1d4
#9 0xffffffff80bc6f23 at fork_exit+0x83
#10 0xffffffff81169aee at fork_trampoline+0xe
Uptime: 22h14m9s
Dumping 962 out of 8077 MB:..2%..12%..22%..32%..42%..52%..62%..72%..82%..92%

__curthread () at ./machine/pcpu.h:230
230             __asm("movq %%gs:%P1,%0" : "=r" (td) : "n" (OFFSETOF_CURTHREAD));
(kgdb) bt
#0  __curthread () at ./machine/pcpu.h:230
#1  doadump (textdump=<optimized out>) at /usr/src/sys/kern/kern_shutdown.c:366
#2  0xffffffff80c06d2b in kern_reboot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:446
#3  0xffffffff80c071a3 in vpanic (fmt=<optimized out>, ap=0xfffffe00004647c0) at /usr/src/sys/kern/kern_shutdown.c:872
#4  0xffffffff80c06f93 in panic (fmt=<unavailable>) at /usr/src/sys/kern/kern_shutdown.c:799
#5  0xffffffff8118d9ff in trap_fatal (frame=0xfffffe00004649b0, eva=2832) at /usr/src/sys/amd64/amd64/trap.c:929
#6  0xffffffff8118da59 in trap_pfault (frame=0xfffffe00004649b0, usermode=0) at /usr/src/sys/amd64/amd64/trap.c:765
#7  0xffffffff8118d07e in trap (frame=0xfffffe00004649b0) at /usr/src/sys/amd64/amd64/trap.c:441
#8  <signal handler called>
#9  ieee80211_ratectl_tx_complete (ni=<optimized out>, status=<optimized out>) at /usr/src/sys/net80211/ieee80211_ratectl.h:143
#10 rt2860_drain_stats_fifo (sc=0xfffffe004ab76000) at /usr/src/sys/dev/ral/rt2860.c:1122
#11 0xffffffff80901693 in rt2860_intr (arg=0xfffffe004ab76000) at /usr/src/sys/dev/ral/rt2860.c:1417
#12 0xffffffff80bca024 in intr_event_execute_handlers (p=<optimized out>, ie=<optimized out>) at /usr/src/sys/kern/kern_intr.c:1119
#13 ithread_execute_handlers (p=<optimized out>, ie=<optimized out>) at /usr/src/sys/kern/kern_intr.c:1132
#14 ithread_loop (arg=<optimized out>) at /usr/src/sys/kern/kern_intr.c:1212
#15 0xffffffff80bc6f23 in fork_exit (callout=0xffffffff80bc9e50 <ithread_loop>, arg=0xfffff80003652940, frame=0xfffffe0000464c00) at /usr/src/sys/kern/kern_fork.c:1059
#16 <signal handler called>
(kgdb) frame 9
#9  ieee80211_ratectl_tx_complete (ni=<optimized out>, status=<optimized out>) at /usr/src/sys/net80211/ieee80211_ratectl.h:143
143             vap->iv_rate->ir_tx_complete(ni, status);
(kgdb) frame 10
#10 rt2860_drain_stats_fifo (sc=0xfffffe004ab76000) at /usr/src/sys/dev/ral/rt2860.c:1122
1122                            ieee80211_ratectl_tx_complete(ni, txs);
(kgdb) frame 11
#11 0xffffffff80901693 in rt2860_intr (arg=0xfffffe004ab76000) at /usr/src/sys/dev/ral/rt2860.c:1417
1417                    rt2860_drain_stats_fifo(sc);
(kgdb) frame 12
#12 0xffffffff80bca024 in intr_event_execute_handlers (p=<optimized out>, ie=<optimized out>) at /usr/src/sys/kern/kern_intr.c:1119
1119                    ih->ih_handler(ih->ih_argument);
(kgdb)
Comment 1 Eugene Grosbein freebsd_committer 2019-03-16 19:21:18 UTC
Created attachment 202920 [details]
Add wlock for priv modification to ng_iface_shutdown()

The problem seems to be in ng_iface_shutdown() that passes zero ifp to if_detach().

Leaving aside bad memory, the only case in which that can be possible is second call to ng_iface_shutdown() after it nullified priv->ifp and before it destoyed the rest.

Sergey, please apply attached patch and add options INVARIANTS to your kernel, too.

If it does not fix your problem, please post your kernel config file and mpd.conf.
Comment 2 Eugene Grosbein freebsd_committer 2019-03-19 12:47:05 UTC
Comment on attachment 202920 [details]
Add wlock for priv modification to ng_iface_shutdown()

Sorry, the patch not meant for this PR.
Comment 3 Eugene Grosbein freebsd_committer 2019-03-19 12:54:38 UTC
The panic is in ral(4) driver for wireless NIC. Please describe your hardware including output of dmesg and pciconf -lvvv and show your wifi configuration.
Comment 4 Sergey Anokhin 2019-03-19 14:24:53 UTC
(In reply to Eugene Grosbein from comment #3)

Will it enough? Please let me know if you want to see additional info

~# pciconf -lvvv
hostb0@pci0:0:0:0:      class=0x060000 card=0x820b1043 chip=0x29a08086 rev=0x02 hdr=0x00
    vendor     = 'Intel Corporation'
    device     = '82P965/G965 Memory Controller Hub'
    class      = bridge
    subclass   = HOST-PCI
pcib1@pci0:0:1:0:       class=0x060400 card=0x00008086 chip=0x29a18086 rev=0x02 hdr=0x01
    vendor     = 'Intel Corporation'
    device     = '82P965/G965 PCI Express Root Port'
    class      = bridge
    subclass   = PCI-PCI
vgapci0@pci0:0:2:0:     class=0x030000 card=0x820b1043 chip=0x29a28086 rev=0x02 hdr=0x00
    vendor     = 'Intel Corporation'
    device     = '82G965 Integrated Graphics Controller'
    class      = display
    subclass   = VGA
none0@pci0:0:3:0:       class=0x078000 card=0x820b1043 chip=0x29a48086 rev=0x02 hdr=0x00
    vendor     = 'Intel Corporation'
    device     = '82P965/G965 HECI Controller'
    class      = simple comms
uhci0@pci0:0:26:0:      class=0x0c0300 card=0x81ec1043 chip=0x28348086 rev=0x02 hdr=0x00
    vendor     = 'Intel Corporation'
    device     = '82801H (ICH8 Family) USB UHCI Controller'
    class      = serial bus
    subclass   = USB
uhci1@pci0:0:26:1:      class=0x0c0300 card=0x81ec1043 chip=0x28358086 rev=0x02 hdr=0x00
    vendor     = 'Intel Corporation'
    device     = '82801H (ICH8 Family) USB UHCI Controller'
    class      = serial bus
    subclass   = USB
ehci0@pci0:0:26:7:      class=0x0c0320 card=0x81ec1043 chip=0x283a8086 rev=0x02 hdr=0x00
    vendor     = 'Intel Corporation'
    device     = '82801H (ICH8 Family) USB2 EHCI Controller'
    class      = serial bus
    subclass   = USB
hdac0@pci0:0:27:0:      class=0x040300 card=0x821a1043 chip=0x284b8086 rev=0x02 hdr=0x00
    vendor     = 'Intel Corporation'
    device     = '82801H (ICH8 Family) HD Audio Controller'
    class      = multimedia
    subclass   = HDA
pcib2@pci0:0:28:0:      class=0x060400 card=0x821a1043 chip=0x283f8086 rev=0x02 hdr=0x01
    vendor     = 'Intel Corporation'
    device     = '82801H (ICH8 Family) PCI Express Port 1'
    class      = bridge
    subclass   = PCI-PCI
pcib3@pci0:0:28:4:      class=0x060400 card=0x821a1043 chip=0x28478086 rev=0x02 hdr=0x01
    vendor     = 'Intel Corporation'
    device     = '82801H (ICH8 Family) PCI Express Port 5'
    class      = bridge
    subclass   = PCI-PCI
uhci2@pci0:0:29:0:      class=0x0c0300 card=0x81ec1043 chip=0x28308086 rev=0x02 hdr=0x00
    vendor     = 'Intel Corporation'
    device     = '82801H (ICH8 Family) USB UHCI Controller'
    class      = serial bus
    subclass   = USB
uhci3@pci0:0:29:1:      class=0x0c0300 card=0x81ec1043 chip=0x28318086 rev=0x02 hdr=0x00
    vendor     = 'Intel Corporation'
    device     = '82801H (ICH8 Family) USB UHCI Controller'
    class      = serial bus
    subclass   = USB
uhci4@pci0:0:29:2:      class=0x0c0300 card=0x81ec1043 chip=0x28328086 rev=0x02 hdr=0x00
    vendor     = 'Intel Corporation'
    device     = '82801H (ICH8 Family) USB UHCI Controller'
    class      = serial bus
    subclass   = USB
ehci1@pci0:0:29:7:      class=0x0c0320 card=0x81ec1043 chip=0x28368086 rev=0x02 hdr=0x00
    vendor     = 'Intel Corporation'
    device     = '82801H (ICH8 Family) USB2 EHCI Controller'
    class      = serial bus
    subclass   = USB
pcib4@pci0:0:30:0:      class=0x060401 card=0x821a1043 chip=0x244e8086 rev=0xf2 hdr=0x01
    vendor     = 'Intel Corporation'
    device     = '82801 PCI Bridge'
    class      = bridge
    subclass   = PCI-PCI
isab0@pci0:0:31:0:      class=0x060100 card=0x821a1043 chip=0x28108086 rev=0x02 hdr=0x00
    vendor     = 'Intel Corporation'
    device     = '82801HB/HR (ICH8/R) LPC Interface Controller'
    class      = bridge
    subclass   = PCI-ISA
ahci1@pci0:0:31:2:      class=0x010601 card=0x821a1043 chip=0x28248086 rev=0x02 hdr=0x00
    vendor     = 'Intel Corporation'
    device     = '82801HB (ICH8) 4 port SATA Controller [AHCI mode]'
    class      = mass storage
    subclass   = SATA
none1@pci0:0:31:3:      class=0x0c0500 card=0x821a1043 chip=0x283e8086 rev=0x02 hdr=0x00
    vendor     = 'Intel Corporation'
    device     = '82801H (ICH8 Family) SMBus Controller'
    class      = serial bus
    subclass   = SMBus
ahci0@pci0:2:0:0:       class=0x010601 card=0x81e41043 chip=0x2363197b rev=0x03 hdr=0x00
    vendor     = 'JMicron Technology Corp.'
    device     = 'JMB363 SATA/IDE Controller'
    class      = mass storage
    subclass   = SATA
atapci0@pci0:2:0:1:     class=0x010185 card=0x81e41043 chip=0x2363197b rev=0x03 hdr=0x00
    vendor     = 'JMicron Technology Corp.'
    device     = 'JMB363 SATA/IDE Controller'
    class      = mass storage
    subclass   = ATA
rl0@pci0:4:0:0: class=0x020000 card=0x813910ec chip=0x813910ec rev=0x10 hdr=0x00
    vendor     = 'Realtek Semiconductor Co., Ltd.'
    device     = 'RTL-8100/8101L/8139 PCI Fast Ethernet Adapter'
    class      = network
    subclass   = ethernet
ral0@pci0:4:1:0:        class=0x028000 card=0x3c051186 chip=0x53601814 rev=0x00 hdr=0x00
    vendor     = 'Ralink corp.'
    device     = 'RT5360 Wireless 802.11n 1T/1R'
    class      = network
none2@pci0:4:3:0:       class=0x0c0010 card=0x81fe1043 chip=0x30441106 rev=0xc0 hdr=0x00
    vendor     = 'VIA Technologies, Inc.'
    device     = 'VT6306/7/8 [Fire II(M)] IEEE 1394 OHCI Controller'
    class      = serial bus
    subclass   = FireWire
skc0@pci0:4:4:0:        class=0x020000 card=0x811a1043 chip=0x432011ab rev=0x14 hdr=0x00
    vendor     = 'Marvell Technology Group Ltd.'
    device     = '88E8001 Gigabit Ethernet Controller'
    class      = network
    subclass   = ethernet

# cat /etc/hostapd.conf
interface=wlan0
debug=1
ctrl_interface=/var/run/hostapd
ctrl_interface_group=wheel
ssid=PAPA
wpa=2
wpa_passphrase=<password>
wpa_key_mgmt=WPA-PSK
wpa_pairwise=TKIP CCMP
channel=3
hw_mode=g
wmm_enabled=1

logger_syslog=-1
logger_syslog_level=2
logger_stdout=-1
logger_stdout_level=2

# ifconfig
rl0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=2008<VLAN_MTU,WOL_MAGIC>
        ether 56:00:ff:aa:ce:fe
        inet 192.168.1.1 netmask 0xffffff00 broadcast 192.168.1.255
        media: Ethernet autoselect (100baseTX <full-duplex>)
        status: active
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
sk0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=80009<RXCSUM,VLAN_MTU,LINKSTATE>
        ether 00:1b:fc:09:fb:60
        inet 10.0.0.1 netmask 0xffffff00 broadcast 10.0.0.255
        media: Ethernet autoselect (100baseTX <full-duplex>)
        status: active
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
        inet6 ::1 prefixlen 128
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3
        inet 127.0.0.1 netmask 0xff000000
        groups: lo
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
enc0: flags=0<> metric 0 mtu 1536
        groups: enc
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
pfsync0: flags=0<> metric 0 mtu 1500
        syncpeer: 0.0.0.0 maxupd: 128 defer: off
        groups: pfsync
pflog0: flags=0<> metric 0 mtu 33160
        groups: pflog
wlan0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
        ether ac:f1:df:0b:14:46
        groups: wlan
        ssid PAPA channel 3 (2422 MHz 11g) bssid ac:f1:df:0b:14:46
        regdomain FCC country US authmode WPA2/802.11i privacy MIXED
        deftxkey 2 TKIP 2:128-bit TKIP 3:128-bit txpower 30 scanvalid 60
        protmode CTS wme dtimperiod 1 -dfs
        media: IEEE 802.11 Wireless Ethernet autoselect mode 11g <hostap>
        status: running
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        ether 02:41:b2:ac:b6:00
        id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
        maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
        root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
        member: wlan0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 7 priority 128 path cost 370370
        member: rl0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 1 priority 128 path cost 55
        groups: bridge
        nd6 options=9<PERFORMNUD,IFDISABLED>
ng0: flags=88d1<UP,POINTOPOINT,RUNNING,NOARP,SIMPLEX,MULTICAST> metric 0 mtu 1462
        inet <external IP> --> <provider IP> netmask 0xffffffff
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
tun1: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> metric 0 mtu 1500
        options=80000<LINKSTATE>
        inet6 fe80::21b:fcff:fe09:fb60%tun1 prefixlen 64 scopeid 0xa
        inet 10.1.200.1 --> 10.1.200.2 netmask 0xffffff00
        groups: tun
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
        Opened by PID 2175
tun0: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> metric 0 mtu 1500
        options=80000<LINKSTATE>
        inet6 fe80::21b:fcff:fe09:fb60%tun0 prefixlen 64 scopeid 0xb
        inet 10.20.0.5 --> 10.20.0.1 netmask 0xffffff00
        groups: tun
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
        Opened by PID 2192
Comment 5 Sergey Anokhin 2019-03-19 14:35:54 UTC
Part of etc.conf

ifconfig_sk0="inet 10.0.0.1 netmask 255.255.255.0"
cloned_interfaces="bridge0"
ifconfig_rl0="inet 192.168.1.1 netmask 255.255.255.0"
ifconfig_bridge0="addm rl0 addm wlan0 up"
ifconfig_wlan0="up"
hostapd_enable="YES"
wlans_ral0="wlan0"
create_args_wlan0="wlanmode hostap"
hostapd_enable="YES"