Bug 269842 - iwlwifi: crash (kernel panic) on ifconfig wlan0 destroy
Summary: iwlwifi: crash (kernel panic) on ifconfig wlan0 destroy
Status: Closed FIXED
Alias: None
Product: Base System
Classification: Unclassified
Component: kern (show other bugs)
Version: CURRENT
Hardware: Any Any
: --- Affects Some People
Assignee: Bjoern A. Zeeb
URL:
Keywords: crash
Depends on:
Blocks: iwlwifi
  Show dependency treegraph
 
Reported: 2023-02-26 21:25 UTC by Michael Zhilin
Modified: 2024-01-10 07:09 UTC (History)
4 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Michael Zhilin freebsd_committer freebsd_triage 2023-02-26 21:25:40 UTC
Crash on fresh CURRENT main-n261047-58e3631bd08c: Tue Feb 21 17:09:40 MSK 2023

[170799] iwlwifi0: Microcode SW error detected. Restarting 0x0.
[170799] iwlwifi0: Start IWL Error Log Dump:
[170799] iwlwifi0: Transport status: 0x0000004B, valid: 6
[170799] iwlwifi0: Loaded firmware version: 73.35c0a2c6.0 cc-a0-73.ucode
[170799] iwlwifi0: 0x00000071 | NMI_INTERRUPT_UMAC_FATAL    
[170799] iwlwifi0: 0x4000A2F0 | trm_hw_status0
[170799] iwlwifi0: 0x00000000 | trm_hw_status1
[170799] iwlwifi0: 0x004FBD66 | branchlink2
[170799] iwlwifi0: 0x00015074 | interruptlink1
[170799] iwlwifi0: 0x00015074 | interruptlink2
[170799] iwlwifi0: 0x00004D30 | data1
[170799] iwlwifi0: 0x00001000 | data2
[170799] iwlwifi0: 0x00000000 | data3
[170799] iwlwifi0: 0x89008A52 | beacon time
[170799] iwlwifi0: 0x1F7AC5D1 | tsf low
[170799] iwlwifi0: 0x00000114 | tsf hi
[170799] iwlwifi0: 0x00000000 | time gp1
[170799] iwlwifi0: 0x0D180284 | time gp2
[170799] iwlwifi0: 0x00000001 | uCode revision type
[170799] iwlwifi0: 0x00000049 | uCode version major
[170799] iwlwifi0: 0x35C0A2C6 | uCode version minor
[170799] iwlwifi0: 0x00000340 | hw version
[170799] iwlwifi0: 0x00C89000 | board version
[170799] iwlwifi0: 0x809AF503 | hcmd
[170799] iwlwifi0: 0x62EA8000 | isr0
[170799] iwlwifi0: 0x00000000 | isr1
[170799] iwlwifi0: 0x08F00002 | isr2
[170799] iwlwifi0: 0x00C000C9 | isr3
[170799] iwlwifi0: 0x00000000 | isr4
[170799] iwlwifi0: 0x01E7001C | last cmd Id
[170799] iwlwifi0: 0x00004D30 | wait_event
[170799] iwlwifi0: 0x00000854 | l2p_control
[170799] iwlwifi0: 0x00000000 | l2p_duration
[170799] iwlwifi0: 0x0000000F | l2p_mhvalid
[170799] iwlwifi0: 0x00000000 | l2p_addr_match
[170799] iwlwifi0: 0x00000009 | lmpm_pmg_sel
[170799] iwlwifi0: 0x00000000 | timestamp
[170799] iwlwifi0: 0x00001050 | flow_handler
[170799] iwlwifi0: Start IWL Error Log Dump:
[170799] iwlwifi0: Transport status: 0x0000004B, valid: 7
[170799] iwlwifi0: 0x20003418 | ADVANCED_SYSASSERT
[170799] iwlwifi0: 0x00000000 | umac branchlink1
[170799] iwlwifi0: 0x80455D6C | umac branchlink2
[170799] iwlwifi0: 0x80455BE8 | umac interruptlink1
[170799] iwlwifi0: 0x00000000 | umac interruptlink2
[170799] iwlwifi0: 0x00000000 | umac data1
[170799] iwlwifi0: 0x00000001 | umac data2
[170799] iwlwifi0: 0x00000000 | umac data3
[170799] iwlwifi0: 0x00000049 | umac major
[170799] iwlwifi0: 0x35C0A2C6 | umac minor
[170799] iwlwifi0: 0x0D18027E | frame pointer
[170799] iwlwifi0: 0xC0886B5C | stack pointer
[170799] iwlwifi0: 0x00670128 | last host cmd
[170799] iwlwifi0: 0x00000000 | isr status reg
[170799] iwlwifi0: IML/ROM dump:
[170799] iwlwifi0: 0x00000003 | IML/ROM error/state
[170799] iwlwifi0: 0x00006149 | IML/ROM data1
[170799] iwlwifi0: 0x00000080 | IML/ROM WFPM_AUTH_KEY_0
[170799] iwlwifi0: Fseq Registers:
[170799] iwlwifi0: 0x60000000 | FSEQ_ERROR_CODE
[170799] iwlwifi0: 0x80290021 | FSEQ_TOP_INIT_VERSION
[170799] iwlwifi0: 0x00050008 | FSEQ_CNVIO_INIT_VERSION
[170799] iwlwifi0: 0x0000A503 | FSEQ_OTP_VERSION
[170799] iwlwifi0: 0x80000003 | FSEQ_TOP_CONTENT_VERSION
[170799] iwlwifi0: 0x4552414E | FSEQ_ALIVE_TOKEN
[170799] iwlwifi0: 0x00100530 | FSEQ_CNVI_ID
[170799] iwlwifi0: 0x00000532 | FSEQ_CNVR_ID
[170799] iwlwifi0: 0x00100530 | CNVI_AUX_MISC_CHIP
[170799] iwlwifi0: 0x00000532 | CNVR_AUX_MISC_CHIP
[170799] iwlwifi0: 0x05B0905B | CNVR_SCU_SD_REGS_SD_REG_DIG_DCDC_VTRIM
[170799] iwlwifi0: 0x0000025B | CNVR_SCU_SD_REGS_SD_REG_ACTIVE_VDIG_MIRROR
[170799] iwlwifi0: WRT: Collecting data: ini trigger 4 fired (delay=0ms).
[170799] iwlwifi0: FW error in SYNC CMD MAC_CONTEXT_CMD
[170799] #0 0xffffffff80e5c4db at linux_dump_stack+0x1b
[170799] #1 0xffffffff8441b0e4 at iwl_trans_txq_send_hcmd+0x414
[170799] #2 0xffffffff843c317e at iwl_trans_send_cmd+0xce
[170799] #3 0xffffffff84402b2f at iwl_mvm_send_cmd_pdu+0x6f
[170799] #4 0xffffffff843d31ad at iwl_mvm_mac_ctxt_remove+0x7d
[170799] #5 0xffffffff843d6e8c at iwl_mvm_mac_remove_interface+0x1ac
[170799] #6 0xffffffff80e56bd5 at lkpi_80211_mo_remove_interface+0x65
[170799] #7 0xffffffff80e54015 at lkpi_ic_vap_delete+0xb5
[170799] #8 0xffffffff80d1b249 at if_clone_destroyif+0x239
[170799] #9 0xffffffff80d1af94 at if_clone_destroy+0x114
[170799] #10 0xffffffff80d17cd1 at ifioctl+0x741
[170799] #11 0xffffffff80c61462 at kern_ioctl+0x202
[170799] #12 0xffffffff80c611aa at sys_ioctl+0x12a
[170799] #13 0xffffffff810c93de at amd64_syscall+0x12e
[170799] #14 0xffffffff8109c5eb at fast_syscall_common+0xf8
[170799] iwlwifi0: Failed to remove MAC context: -5
<118>[170799] 2022-12-06T08:00:29.934845+03:00 tamagawa.unclebear.ru dhclient[23576] receive_packet failed on wlan0: Device not configured
<118>[170799] 2022-12-06T08:00:29.934925+03:00 tamagawa.unclebear.ru dhclient[23576] ioctl(SIOCGIFFLAGS) on wlan0: Operation not permitted
<118>[170799] 2022-12-06T08:00:29.934980+03:00 tamagawa.unclebear.ru dhclient[23576] Interface wlan0 no longer appears valid.
<118>2022-12-06T08:00:29.935037+03:00 tamagawa.unclebear.ru dhclient[23576] No live interfaces to poll on - exiting.
<118>[170799] 2022-12-06T08:00:29.935192+03:00 tamagawa.unclebear.ru dhclient[23576] exiting.
<118>[170799] 2022-12-06T08:00:29.935568+03:00 tamagawa.unclebear.ru dhclient[23576] connection closed
<118>[170799] 2022-12-06T08:00:29.935625+03:00 tamagawa.unclebear.ru dhclient[23576] exiting.
<6>[170799] pid 23262 (wpa_supplicant), jid 0, uid 0: exited on signal 11 (core dumped)
[170800] iwlwifi0: Failed to synchronize multicast groups update
[170800] iwlwifi0: Failed to send flush command (-5)
[170800] iwlwifi0: flush request fail
[170800] iwlwifi0: Couldn't send the SESSION_PROTECTION_CMD
<6>[170800] wlan0: link state changed to DOWN
[170800] iwlwifi0: iwl_trans_wait_txq_empty bad state = 0
[170800] iwlwifi0: Failed to trigger RX queues sync (-5)
[170800] iwlwifi0: Failed to send MAC context (action:2): -5
[170800] iwlwifi0: Failed to synchronize multicast groups update
[170800] iwlwifi0: Failed to send MAC context (action:2): -5
[170800] iwlwifi0: failed to update MAC 0xfffffe01b46f2c9eM
[170800] iwlwifi0: Failed to remove station. Id=0
[170800] iwlwifi0: failed to remove AP station
[170800] iwlwifi0: Failed to send MAC context (action:2): -5
[170800] iwlwifi0: failed to update MAC 0xfffffe01b46f2c9eM (clear after unassoc)
[170800] iwlwifi0: Failed to synchronize multicast groups update
[170800] iwlwifi0: Failed to send MAC context (action:2): -5
[170800] iwlwifi0: failed to update MAC 0xfffffe01b46f2c9eM
[170800] iwlwifi0: Failed to send binding (action:3): -5
[170800] iwlwifi0: PHY ctxt cmd error. ret=-5
[170800] 
[170800] 
[170800] Fatal trap 9: general protection fault while in kernel mode
[170800] cpuid = 0; apic id = 00
[170800] instruction pointer	= 0x20:0xffffffff80bc4c39
[170800] stack pointer	        = 0x0:0xfffffe0169b4ea10
[170800] frame pointer	        = 0x0:0xfffffe0169b4ea50
[170800] code segment		= base rx0, limit 0xfffff, type 0x1b
[170800] 			= DPL 0, pres 1, long 1, def32 0, gran 1
[170800] processor eflags	= interrupt enabled, resume, IOPL = 0
[170800] current process		= 55330 (ifconfig)
[170800] rdi: deadc0dedeadc0f6 rsi: fffffe0157cab720 rdx:                0
[170800] rcx:              865  r8: ffffffff81b10998  r9:                0
[170800] rax:                1 rbx: fffffe0174406000 rbp: fffffe0169b4ea50
[170800] r10:                0 r11:         8103b345 r12: ffffffff81215df1
[170800] r13:              865 r14: deadc0dedeadc10e r15: fffff80298a5f800
[170800] trap number		= 9
[170800] WARNING !drm_modeset_is_locked(&crtc->mutex) failed at /usr/home/mizhka/repo/fbsd/drm-kmod/drivers/gpu/drm/drm_atomic_helper.c:619
[170800] WARNING !drm_modeset_is_locked(&crtc->mutex) failed at /usr/home/mizhka/repo/fbsd/drm-kmod/drivers/gpu/drm/drm_atomic_helper.c:619
[170800] WARNING !drm_modeset_is_locked(&crtc->mutex) failed at /usr/home/mizhka/repo/fbsd/drm-kmod/drivers/gpu/drm/drm_atomic_helper.c:619
[170800] WARNING !drm_modeset_is_locked(&crtc->mutex) failed at /usr/home/mizhka/repo/fbsd/drm-kmod/drivers/gpu/drm/drm_atomic_helper.c:619
[170800] WARNING !drm_modeset_is_locked(&dev->mode_config.connection_mutex) failed at /usr/home/mizhka/repo/fbsd/drm-kmod/drivers/gpu/drm/drm_atomic_helper.c:669
[170800] WARNING !drm_modeset_is_locked(&dev->mode_config.connection_mutex) failed at /usr/home/mizhka/repo/fbsd/drm-kmod/drivers/gpu/drm/drm_atomic_helper.c:669
[170800] WARNING !drm_modeset_is_locked(&plane->mutex) failed at /usr/home/mizhka/repo/fbsd/drm-kmod/drivers/gpu/drm/drm_atomic_helper.c:894
[170800] WARNING !drm_modeset_is_locked(&plane->mutex) failed at /usr/home/mizhka/repo/fbsd/drm-kmod/drivers/gpu/drm/drm_atomic_helper.c:894
[170800] WARNING !drm_modeset_is_locked(&plane->mutex) failed at /usr/home/mizhka/repo/fbsd/drm-kmod/drivers/gpu/drm/drm_atomic_helper.c:894
[170800] WARNING !drm_modeset_is_locked(&plane->mutex) failed at /usr/home/mizhka/repo/fbsd/drm-kmod/drivers/gpu/drm/drm_atomic_helper.c:894
[170800] WARNING !drm_modeset_is_locked(&plane->mutex) failed at /usr/home/mizhka/repo/fbsd/drm-kmod/drivers/gpu/drm/drm_atomic_helper.c:894
[170800] WARNING !drm_modeset_is_locked(&plane->mutex) failed at /usr/home/mizhka/repo/fbsd/drm-kmod/drivers/gpu/drm/drm_atomic_helper.c:894
[170800] WARNING !drm_modeset_is_locked(&plane->mutex) failed at /usr/home/mizhka/repo/fbsd/drm-kmod/drivers/gpu/drm/drm_atomic_helper.c:894
[170800] WARNING !drm_modeset_is_locked(&plane->mutex) failed at /usr/home/mizhka/repo/fbsd/drm-kmod/drivers/gpu/drm/drm_atomic_helper.c:894
[170800] WARNING !drm_modeset_is_locked(&plane->mutex) failed at /usr/home/mizhka/repo/fbsd/drm-kmod/drivers/gpu/drm/drm_atomic_helper.c:894
[170800]                                 panic: general protection fault
[170800] cpuid = 0
[170800] time = 1670302830
[170800] KDB: stack backtrace:
[170800] db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe0169b4e830
[170800] vpanic() at vpanic+0x151/frame 0xfffffe0169b4e880
[170800] panic() at panic+0x43/frame 0xfffffe0169b4e8e0
[170800] trap_fatal() at trap_fatal+0x409/frame 0xfffffe0169b4e940
[170800] calltrap() at calltrap+0x8/frame 0xfffffe0169b4e940
[170800] --- trap 0x9, rip = 0xffffffff80bc4c39, rsp = 0xfffffe0169b4ea10, rbp = 0xfffffe0169b4ea50 ---
[170800] __mtx_lock_flags() at __mtx_lock_flags+0x49/frame 0xfffffe0169b4ea50
[170800] ieee80211_free_node() at ieee80211_free_node+0x34/frame 0xfffffe0169b4ea90
[170800] ieee80211_node_vdetach() at ieee80211_node_vdetach+0x2b/frame 0xfffffe0169b4eab0
[170800] ieee80211_vap_detach() at ieee80211_vap_detach+0x60e/frame 0xfffffe0169b4eb00
[170800] lkpi_ic_vap_delete() at lkpi_ic_vap_delete+0xca/frame 0xfffffe0169b4eb40
[170800] if_clone_destroyif() at if_clone_destroyif+0x239/frame 0xfffffe0169b4eb90
[170800] if_clone_destroy() at if_clone_destroy+0x114/frame 0xfffffe0169b4ebd0
[170800] ifioctl() at ifioctl+0x741/frame 0xfffffe0169b4ecc0
[170800] kern_ioctl() at kern_ioctl+0x202/frame 0xfffffe0169b4ed30
[170800] sys_ioctl() at sys_ioctl+0x12a/frame 0xfffffe0169b4ee00
[170800] amd64_syscall() at amd64_syscall+0x12e/frame 0xfffffe0169b4ef30
[170800] fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0169b4ef30
[170800] --- syscall (54, FreeBSD ELF64, sys_ioctl), rip = 0x4eab25ba97a, rsp = 0x4eaadf7a4a8, rbp = 0x4eaadf7a4c0 ---
[170800] KDB: enter: panic
Comment 1 Graham Perrin 2023-09-23 16:28:43 UTC
(In reply to Michael Zhilin from comment #0)

Reproducible with src 0a22677bce28 or later in 15.0-CURRENT? 

What's the Wi-Fi hardware in your case? 

(To distinguish this bug 269842 from more recent bug 273985.)
Comment 2 Bjoern A. Zeeb freebsd_committer freebsd_triage 2023-09-30 08:46:40 UTC
if you can try main: please update to/past the revision mentioned in:
https://lists.freebsd.org/archives/freebsd-wireless/2023-September/001441.html
Comment 3 Bjoern A. Zeeb freebsd_committer freebsd_triage 2023-09-30 09:00:03 UTC
This one is different.  I just reproduced.  I'll let you know once it's (hopefully) fixed.
Comment 4 commit-hook freebsd_committer freebsd_triage 2023-09-30 15:29:44 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=dbf7691999abe501e0ebc0fe4d8d9e97718d3890

commit dbf7691999abe501e0ebc0fe4d8d9e97718d3890
Author:     Bjoern A. Zeeb <bz@FreeBSD.org>
AuthorDate: 2023-09-30 15:20:53 +0000
Commit:     Bjoern A. Zeeb <bz@FreeBSD.org>
CommitDate: 2023-09-30 15:27:05 +0000

    LinuxKPI: 802.11: improve lkpi_ic_vap_delete()

    Changing the order of cleanup in lkpi_ic_vap_delete() will avoid
    firmware crashes or a direct panic as observed as ieee80211_vap_detach()
    will do most cleanup for us and get us into the correct state.

    The solution is not fully developed yet and one may not be able
    to create a new vap (lkpi_ic_vap_create may fail with error 17)
    afterwards.  It is supposed to be a step towards more stability
    for now.

    Sponsored by:   The FreeBSD Foundation
    MFC after:      3 days
    PR:             269842

 sys/compat/linuxkpi/common/src/linux_80211.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
Comment 5 Bjoern A. Zeeb freebsd_committer freebsd_triage 2023-09-30 15:30:27 UTC
(In reply to commit-hook from comment #4)

Please note that for main I had to apply
https://reviews.freebsd.org/D42027
first as otherwise I would get a panic there.
Comment 6 commit-hook freebsd_committer freebsd_triage 2023-10-04 15:24:36 UTC
A commit in branch stable/14 references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=53ef38313a6dd70e5d929a3d014cc62db807793f

commit 53ef38313a6dd70e5d929a3d014cc62db807793f
Author:     Bjoern A. Zeeb <bz@FreeBSD.org>
AuthorDate: 2023-09-30 15:20:53 +0000
Commit:     Bjoern A. Zeeb <bz@FreeBSD.org>
CommitDate: 2023-10-04 15:19:18 +0000

    LinuxKPI: 802.11: improve lkpi_ic_vap_delete()

    Changing the order of cleanup in lkpi_ic_vap_delete() will avoid
    firmware crashes or a direct panic as observed as ieee80211_vap_detach()
    will do most cleanup for us and get us into the correct state.

    Calling mo_stop() in lkpi_ic_vap_delete() seems to stop the problem
    that further VAPs could not be created (error 17) after the first one
    being destroyed.

    Sponsored by:   The FreeBSD Foundation
    PR:             269842

    (cherry picked from commit dbf7691999abe501e0ebc0fe4d8d9e97718d3890)
    (cherry picked from commit 6c38c6b1b917957d420902213f318bf0153214f2)

 sys/compat/linuxkpi/common/src/linux_80211.c | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)
Comment 7 commit-hook freebsd_committer freebsd_triage 2023-10-09 23:17:27 UTC
A commit in branch releng/14.0 references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=914a319cd6ec26415459259d3f7c5c49d11f0654

commit 914a319cd6ec26415459259d3f7c5c49d11f0654
Author:     Bjoern A. Zeeb <bz@FreeBSD.org>
AuthorDate: 2023-09-30 15:20:53 +0000
Commit:     Bjoern A. Zeeb <bz@FreeBSD.org>
CommitDate: 2023-10-09 23:14:52 +0000

    LinuxKPI: 802.11: improve lkpi_ic_vap_delete()

    Changing the order of cleanup in lkpi_ic_vap_delete() will avoid
    firmware crashes or a direct panic as observed as ieee80211_vap_detach()
    will do most cleanup for us and get us into the correct state.

    Calling mo_stop() in lkpi_ic_vap_delete() seems to stop the problem
    that further VAPs could not be created (error 17) after the first one
    being destroyed.

    Sponsored by:   The FreeBSD Foundation
    PR:             269842
    Approved by:    re (gjb)

    (cherry picked from commit dbf7691999abe501e0ebc0fe4d8d9e97718d3890)
    (cherry picked from commit 6c38c6b1b917957d420902213f318bf0153214f2)
    (cherry picked from commit 53ef38313a6dd70e5d929a3d014cc62db807793f)

 sys/compat/linuxkpi/common/src/linux_80211.c | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)
Comment 8 Bjoern A. Zeeb freebsd_committer freebsd_triage 2023-10-25 21:26:02 UTC
I believe this specific firmware crash should be fixed.
Could you please report back if this is still a problem in 15 or 14?
Comment 9 commit-hook freebsd_committer freebsd_triage 2023-11-29 16:39:41 UTC
A commit in branch stable/13 references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=3419d14597153c15756d2737d16c168edf33247d

commit 3419d14597153c15756d2737d16c168edf33247d
Author:     Bjoern A. Zeeb <bz@FreeBSD.org>
AuthorDate: 2023-09-30 15:20:53 +0000
Commit:     Bjoern A. Zeeb <bz@FreeBSD.org>
CommitDate: 2023-11-29 16:36:09 +0000

    LinuxKPI: 802.11: improve lkpi_ic_vap_delete()

    Changing the order of cleanup in lkpi_ic_vap_delete() will avoid
    firmware crashes or a direct panic as observed as ieee80211_vap_detach()
    will do most cleanup for us and get us into the correct state.

    The solution is not fully developed yet and one may not be able
    to create a new vap (lkpi_ic_vap_create may fail with error 17)
    afterwards.  It is supposed to be a step towards more stability
    for now.

    Sponsored by:   The FreeBSD Foundation
    PR:             269842

    (cherry picked from commit dbf7691999abe501e0ebc0fe4d8d9e97718d3890)

 sys/compat/linuxkpi/common/src/linux_80211.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
Comment 10 Mark Linimon freebsd_committer freebsd_triage 2024-01-10 07:09:21 UTC
^Triage: committed to all supported branches.