Bug 262150 - iwlwifi: intermittent failure with wpa on Framework laptop, Intel AX210
Summary: iwlwifi: intermittent failure with wpa on Framework laptop, Intel AX210
Status: Closed FIXED
Alias: None
Product: Base System
Classification: Unclassified
Component: kern (show other bugs)
Version: CURRENT
Hardware: Any Any
: --- Affects Only Me
Assignee: Bjoern A. Zeeb
URL:
Keywords: needs-qa
: 262765 (view as bug list)
Depends on:
Blocks: frameworklaptop
  Show dependency treegraph
 
Reported: 2022-02-23 15:02 UTC by Ed Maste
Modified: 2022-04-01 21:33 UTC (History)
5 users (show)

See Also:


Attachments
Full dmesg from start (24.97 KB, text/plain)
2022-03-31 14:12 UTC, Tomoaki AOKI
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Ed Maste freebsd_committer freebsd_triage 2022-02-23 15:02:32 UTC
AX210 8086:2725 on Framework laptop

iwlwifi0 dmesg entries:

Intel(R) Wireless WiFi based driver for FreeBSD
iwlwifi0: <iwlwifi> mem 0x7a200000-0x7a203fff at device 0.0 on pci2
iwlwifi0: successfully loaded firmware image 'iwlwifi-ty-a0-gf-a0-67.ucode'
iwlwifi0: api flags index 2 larger than supported by driver
iwlwifi0: TLV_FW_FSEQ_VERSION: FSEQ Version: 0.0.2.34
iwlwifi0: loaded firmware version 67.8f59b80b.0 ty-a0-gf-a0-67.ucode op_mode iwlmvm
iwlwifi0: Detected Intel(R) Wi-Fi 6 AX210 160MHz, REV=0x420
iwlwifi0: WRT: Failed to set DRAM buffer for alloc id 1, ret=-1
iwlwifi0: WRT: Failed to set DRAM buffer for alloc id 2, ret=-1
iwlwifi0: WRT: Failed to set DRAM buffer for alloc id 3, ret=-1
iwlwifi0: successfully loaded firmware image 'iwlwifi-ty-a0-gf-a0.pnvm'
iwlwifi0: loaded PNVM version 4b50f925
iwlwifi0: Detected RF GF, rfid=0x10d000
iwlwifi0: base HW address: f4:26:79:XX:XX:XX
iwlwifi0: WRT: Failed to set DRAM buffer for alloc id 1, ret=-1
iwlwifi0: WRT: Failed to set DRAM buffer for alloc id 2, ret=-1
iwlwifi0: WRT: Failed to set DRAM buffer for alloc id 3, ret=-1
wlan0: Ethernet address: f4:26:79:XX:XX:XX
lo0: link state changed to UP
iwlwifi0: lkpi_sta_scan_to_auth: waiting for 5 queues to be allocated by driver
wlan0: link state changed to UP
iwlwifi0: No beacon heard and the session protection is over already...
wlan0: link state changed to DOWN
iwlwifi0: Microcode SW error detected. Restarting 0x0.
iwlwifi0: Start IWL Error Log Dump:
iwlwifi0: Transport status: 0x0000004B, valid: 6
iwlwifi0: Loaded firmware version: 67.8f59b80b.0 ty-a0-gf-a0-67.ucode
iwlwifi0: 0x00000071 | NMI_INTERRUPT_UMAC_FATAL    
iwlwifi0: 0x000002F0 | trm_hw_status0
iwlwifi0: 0x00000000 | trm_hw_status1
iwlwifi0: 0x004DAC6A | branchlink2
iwlwifi0: 0x004D108E | interruptlink1
iwlwifi0: 0x004D108E | interruptlink2
iwlwifi0: 0x00006394 | data1
iwlwifi0: 0x00000010 | data2
iwlwifi0: 0x00000000 | data3
iwlwifi0: 0x00000000 | beacon time
iwlwifi0: 0x1D5324D7 | tsf low
iwlwifi0: 0x00000117 | tsf hi
iwlwifi0: 0x00000000 | time gp1
iwlwifi0: 0x004890C7 | time gp2
iwlwifi0: 0x00000001 | uCode revision type
iwlwifi0: 0x00000043 | uCode version major
iwlwifi0: 0x8F59B80B | uCode version minor
iwlwifi0: 0x00000420 | hw version
iwlwifi0: 0x00C89002 | board version
iwlwifi0: 0x8091FF00 | hcmd
iwlwifi0: 0x00020000 | isr0
iwlwifi0: 0x01000000 | isr1
iwlwifi0: 0x48F00002 | isr2
iwlwifi0: 0x00C3408C | isr3
iwlwifi0: 0x00200000 | isr4
iwlwifi0: 0x00380103 | last cmd Id
iwlwifi0: 0x00006394 | wait_event
iwlwifi0: 0x00005ED4 | l2p_control
iwlwifi0: 0x00000000 | l2p_duration
iwlwifi0: 0x00000003 | l2p_mhvalid
iwlwifi0: 0x00000000 | l2p_addr_match
iwlwifi0: 0x00000009 | lmpm_pmg_sel
iwlwifi0: 0x00000000 | timestamp
iwlwifi0: 0x0000D0F4 | flow_handler
iwlwifi0: Start IWL Error Log Dump:
iwlwifi0: Transport status: 0x0000004B, valid: 7
iwlwifi0: 0x20101034 | ADVANCED_SYSASSERT
iwlwifi0: 0x00000000 | umac branchlink1
iwlwifi0: 0x8045E20E | umac branchlink2
iwlwifi0: 0x8047FBA0 | umac interruptlink1
iwlwifi0: 0x00000000 | umac interruptlink2
iwlwifi0: 0x00000005 | umac data1
iwlwifi0: 0x00000000 | umac data2
iwlwifi0: 0xDEADBEEF | umac data3
iwlwifi0: 0x00000043 | umac major
iwlwifi0: 0x8F59B80B | umac minor
iwlwifi0: 0x004890BE | frame pointer
iwlwifi0: 0xC0885F04 | stack pointer
iwlwifi0: 0x0039010C | last host cmd
iwlwifi0: 0x00000000 | isr status reg
iwlwifi0: IML/ROM dump:
iwlwifi0: 0x00000B03 | IML/ROM error/state
iwlwifi0: 0x00008484 | IML/ROM data1
iwlwifi0: 0x00000080 | IML/ROM WFPM_AUTH_KEY_0
iwlwifi0: Fseq Registers:
iwlwifi0: 0x60000000 | FSEQ_ERROR_CODE
iwlwifi0: 0x80440003 | FSEQ_TOP_INIT_VERSION
iwlwifi0: 0x00080009 | FSEQ_CNVIO_INIT_VERSION
iwlwifi0: 0x0000A652 | FSEQ_OTP_VERSION
iwlwifi0: 0x00000002 | FSEQ_TOP_CONTENT_VERSION
iwlwifi0: 0x4552414E | FSEQ_ALIVE_TOKEN
iwlwifi0: 0x00400410 | FSEQ_CNVI_ID
iwlwifi0: 0x00400410 | FSEQ_CNVR_ID
iwlwifi0: 0x00400410 | CNVI_AUX_MISC_CHIP
iwlwifi0: 0x00400410 | CNVR_AUX_MISC_CHIP
iwlwifi0: 0x00009061 | CNVR_SCU_SD_REGS_SD_REG_DIG_DCDC_VTRIM
iwlwifi0: 0x00000061 | CNVR_SCU_SD_REGS_SD_REG_ACTIVE_VDIG_MIRROR
iwlwifi0: WRT: Collecting data: ini trigger 4 fired (delay=0ms).
iwlwifi0: FW error in SYNC CMD ADD_STA
#0 0xffffffff80e07563 at linux_dump_stack+0x23
#1 0xffffffff83871c8f at iwl_trans_txq_send_hcmd+0x3df
#2 0xffffffff8381b01e at iwl_trans_send_cmd+0xce
#3 0xffffffff83858a6b at iwl_mvm_send_cmd_status+0x2b
#4 0xffffffff83858b7f at iwl_mvm_send_cmd_pdu_status+0x4f
#5 0xffffffff83849777 at iwl_mvm_drain_sta+0xb7
#6 0xffffffff83849a2b at iwl_mvm_rm_sta+0x8b
#7 0xffffffff8382ffc6 at iwl_mvm_mac_sta_state+0x7e6
#8 0xffffffff80e01bbe at lkpi_80211_mo_sta_state+0x2e
#9 0xffffffff80dfc5f1 at lkpi_sta_auth_to_scan+0x211
#10 0xffffffff80e00d6b at lkpi_iv_newstate+0x1eb
#11 0xffffffff80d44761 at ieee80211_newstate_cb+0x111
#12 0xffffffff80c18007 at taskqueue_run_locked+0x187
#13 0xffffffff80c19332 at taskqueue_thread_loop+0xc2
#14 0xffffffff80b6d153 at fork_exit+0x83
#15 0xffffffff8105034e at fork_trampoline+0xe
iwlwifi0: Failed to send binding (action:3): -5
iwlwifi0: PHY ctxt cmd error. ret=-5
iwlwifi0: lkpi_iv_newstate: error 37 during state transition 5 (RUN) -> 0 (INIT): iv_newstate already handled.
iwlwifi0: Scan failed! ret -5
iwlwifi0: ERROR: lkpi_ic_scan_start: hw_scan returned -5
iwlwifi0: Scan failed! ret -5
iwlwifi0: ERROR: lkpi_ic_scan_start: hw_scan returned -5
iwlwifi0: Scan failed! ret -5
iwlwifi0: ERROR: lkpi_ic_scan_start: hw_scan returned -5

(last lines repeat every 5 seconds)

Several iterations of "devctl disable iwlwifi0; devctl enable iwlwifi0" did not fix it. Was successful after reboot.
Comment 1 Ed Maste freebsd_committer freebsd_triage 2022-02-23 15:04:00 UTC
May be duplicate of 261075
Comment 2 Ed Maste freebsd_committer freebsd_triage 2022-02-23 15:04:38 UTC
(In reply to Ed Maste from comment #1)
PR261075 comment #6 that is
Comment 3 Ed Maste freebsd_committer freebsd_triage 2022-02-23 15:06:11 UTC
Device:

iwlwifi0@pci0:170:0:0:	class=0x028000 rev=0x1a hdr=0x00 vendor=0x8086 device=0x2725 subvendor=0x8086 subdevice=0x0024
    vendor     = 'Intel Corporation'
    device     = 'Wi-Fi 6 AX210/AX211/AX411 160MHz'
    class      = network
Comment 4 Kubilay Kocak freebsd_committer freebsd_triage 2022-02-23 23:41:54 UTC
^Triage: Ed, are we assigning iwlwifi things to bz, or a broader group/alias for multiple relevent people?
Comment 5 Bjoern A. Zeeb freebsd_committer freebsd_triage 2022-02-24 14:02:38 UTC
iwlwifi you can straight assign to me if not then to wireless.

The one here is well known (see wireless mailing list).
There's a two-sided fix;  the first one will come with the driver update today hopefully, the second is a fix to the state machine which I have a partial one so far and will put that into main if not all before the end of the month hopefully.
Comment 6 Tomoaki AOKI 2022-03-28 14:22:35 UTC
Encountered with Intel(R) Wireless-AC 9560 160MHz, REV=0x312, WITHOUT PANIC.
So possibly different problem as Bug 261075, that panics.

On main, there were too many commits between sane boot and error detected.
Today I got same error on stable/13, updated from git 4f69c575996e (OK) to git 6663718bb496 (NG).

Maybe-essential part of dmesg are as below. They appears after broken update.

iwlwifi0: Microcode SW error detected. Restarting 0x0.
iwlwifi0: Start IWL Error Log Dump:
iwlwifi0: Transport status: 0x0000004B, valid: 6
iwlwifi0: Loaded firmware version: 46.4e1ceb39.0 9000-pu-b0-jf-b0-46.ucode
iwlwifi0: 0x00003520 | ADVANCED_SYSASSERT          
iwlwifi0: 0x00A0A200 | trm_hw_status0
iwlwifi0: 0x00000000 | trm_hw_status1
iwlwifi0: 0x004895DE | branchlink2
iwlwifi0: 0x00479056 | interruptlink1
iwlwifi0: 0x00000000 | interruptlink2
iwlwifi0: 0xDEADBEEF | data1
iwlwifi0: 0xDEADBEEF | data2
iwlwifi0: 0xDEADBEEF | data3
iwlwifi0: 0x00000000 | beacon time
iwlwifi0: 0x00618D12 | tsf low
iwlwifi0: 0x00000000 | tsf hi
iwlwifi0: 0x00000000 | time gp1
iwlwifi0: 0x00618D13 | time gp2
iwlwifi0: 0x00000001 | uCode revision type
iwlwifi0: 0x0000002E | uCode version major
iwlwifi0: 0x4E1CEB39 | uCode version minor
iwlwifi0: 0x00000312 | hw version
iwlwifi0: 0x00489002 | board version
iwlwifi0: 0x8042F404 | hcmd
iwlwifi0: 0x24022000 | isr0
iwlwifi0: 0x01000000 | isr1
iwlwifi0: 0x08201802 | isr2
iwlwifi0: 0x00416CC0 | isr3
iwlwifi0: 0x00000000 | isr4
iwlwifi0: 0x0501001C | last cmd Id
iwlwifi0: 0x0001B568 | wait_event
iwlwifi0: 0x00000010 | l2p_control
iwlwifi0: 0x00018034 | l2p_duration
iwlwifi0: 0x0000003F | l2p_mhvalid
iwlwifi0: 0x000000CF | l2p_addr_match
iwlwifi0: 0x0000000D | lmpm_pmg_sel
iwlwifi0: 0x20120103 | timestamp
iwlwifi0: 0x00004064 | flow_handler
iwlwifi0: Start IWL Error Log Dump:
iwlwifi0: Transport status: 0x0000004B, valid: 7
iwlwifi0: 0x20000070 | NMI_INTERRUPT_LMAC_FATAL
iwlwifi0: 0x00000000 | umac branchlink1
iwlwifi0: 0xC0088BBA | umac branchlink2
iwlwifi0: 0x010195B4 | umac interruptlink1
iwlwifi0: 0xC00851E8 | umac interruptlink2
iwlwifi0: 0x00000800 | umac data1
iwlwifi0: 0xC00851E8 | umac data2
iwlwifi0: 0xDEADBEEF | umac data3
iwlwifi0: 0x0000002E | umac major
iwlwifi0: 0x4E1CEB39 | umac minor
iwlwifi0: 0x00618D2F | frame pointer
iwlwifi0: 0xC0887530 | stack pointer
iwlwifi0: 0x003A0128 | last host cmd
iwlwifi0: 0x00000000 | isr status reg
iwlwifi0: IML/ROM dump:
iwlwifi0: 0x00000000 | IML/ROM error/state
iwlwifi0: 0x00000003 | IML/ROM data1
iwlwifi0: Fseq Registers:
iwlwifi0: 0x8A5F44DB | FSEQ_ERROR_CODE
iwlwifi0: 0x00000000 | FSEQ_TOP_INIT_VERSION
iwlwifi0: 0xCFDBE2F7 | FSEQ_CNVIO_INIT_VERSION
iwlwifi0: 0x0000A384 | FSEQ_OTP_VERSION
iwlwifi0: 0xBDB46C5F | FSEQ_TOP_CONTENT_VERSION
iwlwifi0: 0x232BA404 | FSEQ_ALIVE_TOKEN
iwlwifi0: 0x84A3F03C | FSEQ_CNVI_ID
iwlwifi0: 0xE43BE075 | FSEQ_CNVR_ID
iwlwifi0: 0x01000100 | CNVI_AUX_MISC_CHIP
iwlwifi0: 0x01300202 | CNVR_AUX_MISC_CHIP
iwlwifi0: 0x0000485B | CNVR_SCU_SD_REGS_SD_REG_DIG_DCDC_VTRIM
iwlwifi0: 0xA5A5A5A2 | CNVR_SCU_SD_REGS_SD_REG_ACTIVE_VDIG_MIRROR
iwlwifi0: WRT: Collecting data: ini trigger 4 fired (delay=0ms).
iwlwifi0: FW error in SYNC CMD MAC_CONTEXT_CMD
#0 0xffffffff80e99b23 at linux_dump_stack+0x23
#1 0xffffffff834af844 at iwl_trans_txq_send_hcmd+0x424
#2 0xffffffff83455bde at iwl_trans_send_cmd+0xce
#3 0xffffffff83496259 at iwl_mvm_send_cmd_pdu+0x49
#4 0xffffffff83465b70 at iwl_mvm_mac_ctx_send+0x4b0
#5 0xffffffff8346e6c4 at iwl_mvm_bss_info_changed+0x184
#6 0xffffffff80e8fb68 at lkpi_sta_assoc_to_run+0x2a8
#7 0xffffffff80e9399b at lkpi_iv_newstate+0x1bb
#8 0xffffffff80dd6d0a at ieee80211_newstate_cb+0x18a
#9 0xffffffff80cb0cd7 at taskqueue_run_locked+0x187
#10 0xffffffff80cb1ff2 at taskqueue_thread_loop+0xc2
#11 0xffffffff80c09352 at fork_exit+0x82
#12 0xffffffff810d1d7e at fork_trampoline+0xe
iwlwifi0: Failed to send MAC context (action:2): -5
iwlwifi0: failed to update MAC 0xfffffe010ae9dc9eM
iwlwifi0: mcast filter cmd error. ret=-5
iwlwifi0: Failed to synchronize multicast groups update
iwlwifi0: failed to update power mode
iwlwifi0: mcast filter cmd error. ret=-5
iwlwifi0: Failed to synchronize multicast groups update
WARNING iwl_mvm_enable_beacon_filter(mvm, vif, 0) failed at /usr/src/sys/contrib/dev/iwlwifi/mvm/mac80211.c:3510
iwlwifi0: Failed to send MAC context (action:2): -5
iwlwifi0: Failed to send MAC context (action:2): -5
iwlwifi0: failed to update MAC 0xfffffe010ae9dc9eM
iwlwifi0: mcast filter cmd error. ret=-5
iwlwifi0: Failed to synchronize multicast groups update
iwlwifi0: mcast filter cmd error. ret=-5
iwlwifi0: Failed to synchronize multicast groups update
iwlwifi0: Failed to send flush command (-5)
iwlwifi0: flush request fail
iwlwifi0: Couldn't remove the time event
iwlwifi0: Couldn't send TIME_EVENT_CMD: -5
iwlwifi0: iwl_trans_wait_tx_queues_empty bad state = 0
iwlwifi0: Failed to trigger RX queues sync (-5)
wlan0: link state changed to DOWN
iwlwifi0: Failed to send MAC context (action:2): -5
iwlwifi0: Failed to synchronize multicast groups update
iwlwifi0: Failed to send MAC context (action:2): -5
iwlwifi0: failed to update MAC 0xfffffe010ae9dc9eM
iwlwifi0: Failed to remove station. Id=0
iwlwifi0: failed to remove AP station
iwlwifi0: Failed to send MAC context (action:2): -5
iwlwifi0: failed to update MAC 0xfffffe010ae9dc9eM (clear after unassoc)
iwlwifi0: Failed to synchronize multicast groups update
iwlwifi0: Failed to send MAC context (action:2): -5
iwlwifi0: failed to update MAC 0xfffffe010ae9dc9eM
iwlwifi0: Failed to send binding (action:3): -5
iwlwifi0: PHY ctxt cmd error. ret=-5
iwlwifi0: Scan failed! ret -5
iwlwifi0: ERROR: lkpi_ic_scan_start: hw_scan returned -5

(snip 11 sets of 2 lines above)

iwlwifi0: Failed to synchronize multicast groups update
iwlwifi0: Scan failed! ret -5
iwlwifi0: ERROR: lkpi_ic_scan_start: hw_scan returned -5


And then repeats last 2 lines until I forcibly do `ifconfig wlan0 delete`.


When OK, I saw messages below instead.

iwlwifi0: lkpi_sta_scan_to_auth: waiting for 5 queues to be allocated by driver
iwlwifi0: Couldn't drain frames for staid 0, status 0x8
iwlwifi0: lkpi_iv_newstate: error -5 during state transition 5 (RUN) -> 0 (INIT)
Comment 7 Ed Maste freebsd_committer freebsd_triage 2022-03-30 17:01:37 UTC
I followed up on freebsd-wireless but want to leave a note here too; I updated to 196cfd0b2eb640a63d007a7e9c3a74e4256255f3 (plus my assorted WIP) on my Framework laptop with AX210, and wifi seems to be reliable and stable.
Comment 8 tod.jackson 2022-03-31 02:50:32 UTC
*** Bug 262765 has been marked as a duplicate of this bug. ***
Comment 9 Tomoaki AOKI 2022-03-31 13:12:01 UTC
(In reply to Ed Maste from comment #7)

Unfortunately, I still have the same issue as stable/13 on main git 342056fa1c7a36d90feafd593fb980f98563f32c, which is newer than the one you pointed.
So possibly your WIP codes would be essential to fix.
Comment 10 Bjoern A. Zeeb freebsd_committer freebsd_triage 2022-03-31 13:16:46 UTC
(In reply to Tomoaki AOKI from comment #9)

Your problem from above seems different;  can you show the lines before the error from dmesg?  And we should probably not do that here as it's not related to the framework laptop.  Feel free to drop me a direct email with full dmesg leading up to the error and I'll have a look.
Comment 11 Tomoaki AOKI 2022-03-31 14:12:54 UTC
Created attachment 232845 [details]
Full dmesg from start

Full dmesg with replacing serial No.s and MAC addresses with '*'.
Last 3 or 4 lines would be generated when I forcibly ran `ifconfig wlan0 down -defaultif` as root.
Comment 12 Ed Maste freebsd_committer freebsd_triage 2022-03-31 14:27:13 UTC
(In reply to Tomoaki AOKI from comment #9)
My WIP is a umber of build options, userland changes, etc., but if you want to try exactly what I ran on the framework it is available in my GitHub repo at https://github.com/emaste/freebsd/, commit bffcacba215138307e8a020c24d94913f9c823cf

https://github.com/emaste/freebsd/commit/bffcacba215138307e8a020c24d94913f9c823cf
Comment 13 Bjoern A. Zeeb freebsd_committer freebsd_triage 2022-04-01 19:19:20 UTC
(In reply to Ed Maste from comment #12)

I am exchanging emails with Tomoaki AOKI out of band about his problem.

I missed to mention the PR in the commits (as another one), most notably in d9f59799fc3e7940c47aa674c25994e640eae45e .

Given the other problem is not related to the Framework laptop, do you still want to keep this open Ed?
Comment 14 Ed Maste freebsd_committer freebsd_triage 2022-04-01 19:22:45 UTC
The issues that prompted this PR have now been resolved with commits leading up to / including 196cfd0b2eb640a63d007a7e9c3a74e4256255f3
Comment 15 tod.jackson 2022-04-01 21:33:58 UTC
No worries. I will take it up with my BIOS vendor. The issue is fairly similar, but I think it crashes a little harder on my end. After acpica: Import ACPICA 20220331 I got some new ACPI warnings pointing me in that direction. I might have to tough it out with iwm for now, as it seems to avoid the resume issue. The other issues I reported are resolved, buut since it was locking up I'm not 100% sure what we did that fixed it. Anyway, I will give all the shiny stuff another spin later on. Thanks!