Bug 264238 - wpa_supplicant 2.10 fails to associate to open secondary VAP when primary VAP is WPA
Summary: wpa_supplicant 2.10 fails to associate to open secondary VAP when primary VAP...
Status: Closed FIXED
Alias: None
Product: Base System
Classification: Unclassified
Component: wireless (show other bugs)
Version: 13.1-RELEASE
Hardware: amd64 Any
: --- Affects Many People
Assignee: Cy Schubert
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2022-05-25 15:08 UTC by Jaskie
Modified: 2022-09-16 11:14 UTC (History)
13 users (show)

See Also:


Attachments
test files form 13.0 and 13.1 (6.46 KB, text/plain)
2022-06-03 02:23 UTC, Jaskie
no flags Details
Test file tarball form 13.0 and 13.1 (6.46 KB, application/octet-stream)
2022-06-03 02:40 UTC, Jaskie
no flags Details
Correctly call pcap_next_ex (1.74 KB, patch)
2022-06-06 01:33 UTC, Cy Schubert
no flags Details | Diff
Two patches in current and stable/13 that are not in 13.1 (1.21 KB, patch)
2022-06-06 16:01 UTC, Cy Schubert
no flags Details | Diff
dtrace output (9.70 KB, application/x-xz)
2022-06-14 05:06 UTC, Jaskie
no flags Details
dtrace out2 (9.67 KB, application/x-xz)
2022-06-14 14:59 UTC, Jaskie
no flags Details
coredump from wpa_supplicant (80.36 KB, application/x-xz)
2022-06-15 00:40 UTC, Jaskie
no flags Details
dtrace from wpa v2.10 (23.68 KB, application/x-xz)
2022-06-17 05:53 UTC, Jaskie
no flags Details
Revert driver_bsd.c from wpa 2.10 back to 2.9 (25.06 KB, patch)
2022-06-20 01:20 UTC, Cy Schubert
no flags Details | Diff
A better patch (2.20 KB, patch)
2022-06-20 06:28 UTC, Cy Schubert
no flags Details | Diff
wpa out (1.94 KB, text/plain)
2022-06-20 08:06 UTC, Jaskie
no flags Details
wpa out p1 (3.84 KB, text/plain)
2022-06-20 15:29 UTC, Jaskie
no flags Details
Remove upstream cc79eb725f7b564269619ce4a64be2a80927d392 (548 bytes, patch)
2022-06-21 23:32 UTC, Cy Schubert
no flags Details | Diff
output from wap 2.10 patched (2 patches) (1.89 KB, application/x-xz)
2022-06-22 14:43 UTC, Jaskie
no flags Details
Proof of Concept solution (2.87 KB, patch)
2022-06-24 00:01 UTC, Cy Schubert
no flags Details | Diff
tcpdump beacons from wpa (3.14 KB, application/x-xz)
2022-06-24 14:50 UTC, Jaskie
no flags Details
Try ext IE first (735 bytes, patch)
2022-07-02 05:23 UTC, Cy Schubert
no flags Details | Diff
Uese ext IE only (601 bytes, patch)
2022-07-02 08:52 UTC, Cy Schubert
no flags Details | Diff
patch to have wpa_driver_bsd_associate() check for WLAN_EID_RSN IE before setting WPA (2.24 KB, patch)
2022-07-03 13:50 UTC, J.R. Oldroyd
no flags Details | Diff
patch to have wpa_driver_bsd_associate() check for WLAN_EID_RSN IE before setting WPA (1.96 KB, patch)
2022-07-03 14:02 UTC, J.R. Oldroyd
no flags Details | Diff
patch to have wpa_driver_bsd_associate() check for WLAN_EID_RSN IE before setting WPA (2.29 KB, patch)
2022-07-03 14:34 UTC, J.R. Oldroyd
no flags Details | Diff
patch to have wpa_driver_bsd_associate() check for WLAN_EID_RSN IE before setting WPA (2.98 KB, patch)
2022-07-03 15:57 UTC, J.R. Oldroyd
no flags Details | Diff
patch to have wpa_driver_bsd_associate() check for WLAN_EID_RSN IE before setting WPA (2.92 KB, patch)
2022-07-03 16:53 UTC, J.R. Oldroyd
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Jaskie 2022-05-25 15:08:26 UTC
I used to run 13.0-release on my laptop without any problems connecting to WiFi networks with or without a password. But after upgrading to 13.1 release. Network stops working if I connect to WiFi without a password.
It's a Qualcomm Atheros AR9285 wireless network adapter. My office network is a WiFi without password authentication. Instead, after I connect to it a login site will be opened and I'll have to input my username and password there and then connect. Also, the network remembers so if I reboot or leave and come back soon I don't have to login again, it just connect and work. But none of that works in 13.1. I created a boot environment for 13.0 before upgrade and now if I reboot into 13.0 BE my network still works flawlessly. I confirm /etc/wpa_supplicant.conf, /etc/rc.conf and /etc/sysctl.conf are identical between 13.0 and 13.1.

More information will be added if needed.
Comment 1 Jaskie 2022-05-25 15:50:39 UTC
/etc/wpa_supplicant.conf has lines for the WiFi as:

network={                                                                                                                                                                                  
 ssid="WHU-STU"                                                                                                                                                                           
 key_mgmt=NONE                                                                                                                                                                             
}

which works in 13.0.

ifconfig wlan0 reports:

wlan0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        ether 1c:4b:d6:ca:3e:ac
        groups: wlan
        ssid WHU-STU channel 5 (2432 MHz 11g ht/20) bssid 70:d9:31:0e:2c:00             
        regdomain 96 indoor ecm authmode WPA privacy MIXED deftxkey UNDEF
        txpower 20 bmiss 7 scanvalid 60 protmode CTS ampdulimit 64k shortgi
        -stbctx stbcrx -ldpc -uapsd wme burst roaming MANUAL bintval 300
        parent interface: ath0
        media: IEEE 802.11 Wireless Ethernet DS/1Mbps mode 11ng
        status: associated
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>

Though it says it's connected to my network (WHU-STU), but there is no internet access, nor can I open athentification page which automatically prompts in 13.0 when I open firefox. My orgnization has a adimistrator page for us where we can find out our connected devices, and I cannot find my laptop there either though ifconfig says I am connected. But I suppose this is normal since I am not yet logged in now.
I tried service netif/dhclient restart, none worked.

I boot into 13.0 boot environment and WiFi works again. ifconfig reports:

wlan0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        ether 1c:4b:d6:ca:3e:ac
        inet 10.134.135.167/17 broadcast 10.134.255.255
        groups: wlan
        ssid WHU-STU channel 13 (2472 MHz 11g ht/20) bssid fc:b6:98:f6:12:80
        regdomain 96 indoor ecm authmode OPEN privacy OFF txpower 20 bmiss 7
        scanvalid 60 protmode CTS ampdulimit 64k shortgi -stbctx stbcrx -ldpc
        -uapsd wme burst roaming MANUAL bintval 300
        parent interface: ath0
        media: IEEE 802.11 Wireless Ethernet MCS mode 11ng
        status: associated
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
Comment 2 rkoberman 2022-05-25 20:45:13 UTC
This sounds much like the issue I have been seeing on 13-STABLE for about 5 months. Some systems simply seem to ignore my DHCP DISCOVER and REQUEST packets. If I capture traffic, I see the packets exit my system, but nothing ever comes back. This typically happens when I am at hotels. No problems on my home system.

I have tried both the base system dhclient and the ports one and see no difference. ATM I am getting my service (I'm in a hotel right now) by running a hotspot on my phone and connecting to it.

I've seen one other report of this, but was unsure if it was hardware, so I have not opened a ticket. Looks like you should change this one to "Affects some people".

It's hard for me to test as it works fine at home and I don't have access to compare what other systems are doing when traveling.
Comment 3 rkoberman 2022-05-27 05:50:38 UTC
This looks more like a DHCP issue than a wireless one to me.

I see valid DHCP packets sent out with no response. While I suppose it's possible that it is not tied to wireless. I have no ability to test over a wired link as I never see this on my home system, only when I travel and no wired network is available.

I you are seeing this problem and can try a wired interface, please check to confirm as this may belong in freebsd-net an not freebsd-wireless. Also, can you do a packet capture when DHCP runs?

The following commands should do the trick. I see outgoing packets, but no response or any incomint packetsh. This command will put the interface in promiscuous mode while running.
# tcpdump -i wlan0 -w dhcp-issue.bpf -s0
# service dhclient start wlan0
^C
#

For the record, I am on 13.1-STABLE on a Lenovo L15 ThinkPad. I have tried both the internal AX200 (iwlwifi) and the TPlink Realtek 802.11n NIC (urtwn).h
Comment 4 Jaskie 2022-05-27 14:44:05 UTC
I tried what you suggested. I boot into 13.1 BE, again ifconfig reports wlan0 to be connected, while there is no internet access as tested from Firefox.
I then opened terminal and issued tcpdump -i wlan0 -w dhcp-issue.bpf -s0, there is one line output:

tcpdump: listening on wlan0 link-type EN10MB (Ethernet), capture size 262144 bytes.


And then nothing echoes back. I tried service start dhclient and it says it's running. I restarted it, stopped and started again, I tried opening sites in Firefox. All the time nothing echoes back from tcpdump.
Comment 5 Adrian Chadd freebsd_committer freebsd_triage 2022-05-27 18:33:36 UTC
hi!

please try 13.0 kernel w/ 13.1 userland and 13.1 userland with 13.0 kernel. See if that changes anything.
Comment 6 rkoberman 2022-05-27 20:49:13 UTC
I forgot one important detail.  You wrote a file with the capture. I forgot to tell you how to read it.

"tcpdump -r file-ypu-wrote" will print the details. I use Wireshark  which will disect the packets, but  for this, just a basic dump should do. I suspect you will see a few outbound packets but no responses.

Sorry for the oversight.
Comment 7 Jaskie 2022-05-28 14:54:13 UTC
Sorry I am not sure how to boot with another kernel in boot environment. I chose another kernel and 13.1 BE at boot menu and ended up in a boot like this:

% uname -a
FreeBSD freebsd.asus 13.0-RELEASE-p11 FreeBSD 13.0-RELEASE-p11 #0: Tue Apr  5 18:54:35 UTC 2022     root@amd64-builder.daemonology.net:/usr/obj/usr/src/amd64.amd64/sys/GENERIC amd64
% freebsd-version -kru
13.1-RELEASE
13.0-RELEASE-p11
13.1-RELEASE

Network is not working in this boot.  I am not what this is a proper test.

But I did notice there was a 30 seconds wait at boot process saying:

Waiting 30s for the default toute interface..........

But in 13.0 BE there is no such message and delay.

Also, this is from 13.1 BE tcpdump -r dhcp-issue.bfp, output, network not working test output:

reading from file dhcp-issue.bfp, link-type EN10MB (Ethernet)
22:31:38.985986 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from 1c:4b:d6:ca:3e:ac (oui Unknown), length 300
22:31:42.992020 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from 1c:4b:d6:ca:3e:ac (oui Unknown), length 300
22:31:48.996586 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from 1c:4b:d6:ca:3e:ac (oui Unknown), length 300
22:32:04.002978 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from 1c:4b:d6:ca:3e:ac (oui Unknown), length 300
22:32:20.017231 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from 1c:4b:d6:ca:3e:ac (oui Unknown), length 300
22:32:37.090318 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from 1c:4b:d6:ca:3e:ac (oui Unknown), length 300
22:32:44.701679 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from 1c:4b:d6:ca:3e:ac (oui Unknown), length 300
22:32:49.797947 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from 1c:4b:d6:ca:3e:ac (oui Unknown), length 300
22:32:54.801007 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from 1c:4b:d6:ca:3e:ac (oui Unknown), length 300
22:33:02.881961 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from 1c:4b:d6:ca:3e:ac (oui Unknown), length 300
22:33:13.883794 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from 1c:4b:d6:ca:3e:ac (oui Unknown), length 300
22:33:22.892983 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from 1c:4b:d6:ca:3e:ac (oui Unknown), length 300
Comment 8 rkoberman 2022-05-28 21:24:28 UTC
(In reply to Jaskie from comment #7)
You are seeing exactly what I see. Requests go out but there are no responses from the DHCP server. This certainly looks to me like a DHCP problem and not an interface issue.

What I hope to find is someone who can connect a wired interface to their router to see if that works. It really looks to me like some newer equipment is not happy with the REQUEST/ DISCOVER packets sent by either the base or the ports dhclient.

One thing you might try is to go back to dhclient 4a8c3cd0dffb9ae8d6fcec841979aff66a60e967, the first update to releng13.1,  and, if that works, bisect to see where it broke. It is possible that going back that far might fail to build, but that is unlikely. As it is in userland, you only have to build and install /sbin/dhclient and not need to build the kernel or the rest of user software.

It might also be useful to know what router you are using including the firmware version running on it. I can say that everything works fine on my home ASUS RT-ARCH13 running 3.0.0.4.382-52134.
Comment 9 Jaskie 2022-05-29 01:04:06 UTC
(In reply to rkoberman from comment #8)

Unfortunately we only have wireless network so I am not able to test a wired connection. It's a campus network environment and I don't know where I can get hold of any hardware spec information regarding the router. 
And I am a fairly new user of FreeBSD. I am not using ports, nor do I know how to revert to a previous version of /sbin/dhclient. But I do want to help and provide information as long as I can.
Comment 10 Jaskie 2022-05-31 11:30:11 UTC
(In reply to Adrian Chadd from comment #5)

So I mounted 13.1 BE and copied /boot/kernel into my 13.0 BE and vice verse. 13.0 userland with 13.1 kernel, network is fine. But not in 13.1 usernland with 13.0 kernel.


In vanilla 13.0-p11 (network works):

% uname -a
FreeBSD freebsd.asus 13.0-RELEASE-p11 FreeBSD 13.0-RELEASE-p11 #0: Tue Apr  5 18:54:35 UTC 2022     root@amd64-builder.daemonology.net:/usr/obj/usr/src/amd64.amd64/sys/GENERIC  amd64
% freebsd-version -kru
13.0-RELEASE-p11
13.0-RELEASE-p11
13.0-RELEASE-p11

13.0 BE with 13.1 kernel (network works):

% uname -a
FreeBSD freebsd.asus 13.1-RELEASE FreeBSD 13.1-RELEASE releng/13.1-n250148-fc952ac2212 GENERIC  amd64
% freebsd-version -kru
13.0-RELEASE-p11
13.1-RELEASE
13.0-RELEASE-p11

13.1 BE with 13.0-p11 kernel  (network down):

% uname -a
FreeBSD freebsd.asus 13.0-RELEASE-p11 FreeBSD 13.0-RELEASE-p11 #0: Tue Apr  5 18:54:35 UTC 2022     root@amd64-builder.daemonology.net:/usr/obj/usr/src/amd64.amd64/sys/GENERIC amd64
% freebsd-version -kru
13.1-RELEASE
13.0-RELEASE-p11
13.1-RELEASE

During 13.1 with 13.0 kernel boot, there were a lot of 

kldstat: can't stat module id xxx, invalid arhuement

output. 

I noticed there was an old kernel in 13.1 (/boot/kernel.old) and I tried booting 13.1 with this kernel, but it always froze my desktop seconds after I logined in from lightdm, I had to hold power button to force shutdown my laptop. This happened twice.

I was also suggested to run freebsd-update IDS to check my local files, but that seemed to require internet acess too so was not possible for 13.1.
Comment 11 Bjoern A. Zeeb freebsd_committer freebsd_triage 2022-06-01 20:43:23 UTC
(In reply to Jaskie from comment #10)

13.1 user land with 13.0 kernel is not supposed to be supported really though on a stable branch it may mostly work (FreeBSD does not provide forward compatibility -- like most as the future is hard to predict ;-)

The important fact is that a 13.0 user land with a 13.1 kernel works.

In that specific constellation, can you add a /rescue from 13.1 to the 13.0 userland and try to run /rescue13.1/dhclient wlan0 and see if that works.
You may have to disable (SYNC)DHCP from the interface in rc.conf so that the default does not interfere.

If this fails then it would be good to know what dhclient logs to syslog as well (usually /var/log/daemon.log and /var/log/messages).  That might also give a hint.
Comment 12 Jaskie 2022-06-02 02:38:41 UTC
So this is really odd. I today take my laptop with me to my workplace and test 13.1 with another network and it seems the problem is gone. But the two networks are the same, they're both set up by my organization, they don't require a password to connect and do require web login to use. The only difference may be they're from different routers.
I am doing more tests tonight and see what exactly leads to this problem.
Comment 13 rkoberman 2022-06-02 04:58:38 UTC
My laptop works fine at home. It also works with my phone hot spot. It does not work with some routers. It seems to not work in many (most) Marriott hotels. My phone works in the hotel as does my wife's Mac and iPhone.

I still strongly suspect that the issue is that the DHCP client is not working with certain DHCP servers. I'm guessing Cisco, but am only sure in a couple where the login splash tells me that they are using Cisco networking. The hot spots in the last I stayed with this problem was a Ruckus, but, while they can run a dhclient, it is likely a bigger box that handles DHCP assignments for the whole place.

Again, I suspect a compatibility issue between DHCP clients and the DHCP server or some box in between, perhaps a firewall.
Comment 14 Jaskie 2022-06-02 14:36:42 UTC
I can confirm now that network is still not work for the wireless network I reported. Do I close this bug or not? It do seem like compatibility issue with this specific network environment. I don't know how to dig more about this. I am thinking that I might just have to live with the fact that 13.1 just WON'T work for me.
Comment 15 J.R. Oldroyd 2022-06-02 15:36:51 UTC
(In reply to Jaskie from comment #14)

I wouldn't close the PR until the problem is resolved.

What you could do at this point is look at a detailed dump of the packets on both the working and the non-working systems.

Do the tcpdump similar to what was proposed before:
    disable DHCP in /etc/rc.conf (ifconfig_wlan0="..." should not have DHCP in it)
    service netif stop wlan0
    tcpdump -i wlan0 -w file.bpf -s 0
    service netif start wlan0
then view using:
    tcpdump -r file.bpf -vvv port 67 or 68

Make sure that you're using the same bssid both times so that the comparison is useful (e.g., use a "bssid=xx:xx:xx:xx:xx:xx" in your wpa_supplicant.conf).

There has to be some difference between what is sent.  Or there has to be something blocking the DHCP datagrams from reaching the router.  Or blocking the responses.
Comment 16 Bjoern A. Zeeb freebsd_committer freebsd_triage 2022-06-02 16:31:39 UTC
(In reply to J.R. Oldroyd from comment #15)

or some change in certain option handling now makes us send a bad request and depending on server being more lenient or not ..

It's like 5 changes changes in dhclient (if the issue is there):

% git log --oneline freebsd/releng/13.0..freebsd/releng/13.1 sbin/dhclient/
586827df60ce dhclient: support VID 0 (no vlan) decapsulation
75df39760ce0 dhclient: remove patching of static values in BPF programs
3bcf919c4e89 dhclient: skip_to_semi() consumes semicolon already
8751bff1a72e dhclient: support supersede statement for option 54
50ecd99be523 bpf: Add an ioctl to set the VLAN Priority on packets sent by bpf

I would highly prefer to have a binary tcpdump written to a file from both working and non-working setup. Not a textdump of such, so that we can actually inspect packets.


also I am tempted at this point to put this over to freebsd-net as it seems it may not be a wireless issue given kernels can be interchanged (13.0 userland reported to work on 13.1 kernel) and we see associated.  So it's unlikely a broad/multicast wifi problem.
Comment 17 Jaskie 2022-06-03 02:23:45 UTC
Created attachment 234405 [details]
test files form 13.0 and 13.1
Comment 18 Jaskie 2022-06-03 02:35:10 UTC
So I have the same bssid in /etc/wpa_supplicant.conf, 

wlans_ath0="wlan0"
ifconfig_wlan0="WPA"

in /etc/rc.conf in both 13.0 and 13.1. Now I boot up 13.0 or 13.1 network is down and I have to run dhclient wlan0 to connect.

 
I uploaded an attachment file. The two dmesg files are from fresh boot 13.0 and 13.1. Network was not working in both scenarios since DHCP was disabled.  


In 13.0, I boot up and run dhclient and then network is connected instantly

% doas service netif stop wlan0
% doas tcpdump -i wlan0 -w file_13.0_working.bfp -s 0
tcpdump: wlan0: No such device exists                                                                                                            
(BIOCSETIF failed: Device not configured)

So I had to

% doas service netif start wlan0
% doas tcpdump -i wlan0 -w file_13.0_working.bfp -s 0
% doas dhclient wlan0
DHCPREQUEST on wlan0 to 255.255.255.255 port 67
DHCPACK from 10.134.255.254
bound to 10.134.132.85 -- renewal in 3600 seconds.
% ping bing.com   # get ping backs
% Ctrl-C to stop tcpdump

File is uploaded as file_13.0_working.bfp in attatchment.

Then in fresh boot 13.1, network is down. 

% doas dhclient wlan0                                                                                  
DHCPDISCOVER on wlan0 to 255.255.255.255 port 67 interval 4
DHCPDISCOVER on wlan0 to 255.255.255.255 port 67 interval 10
DHCPDISCOVER on wlan0 to 255.255.255.255 port 67 interval 13
DHCPDISCOVER on wlan0 to 255.255.255.255 port 67 interval 9
DHCPDISCOVER on wlan0 to 255.255.255.255 port 67 interval 9
DHCPDISCOVER on wlan0 to 255.255.255.255 port 67 interval 15
No DHCPOFFERS received.
No working leases in persistent database - sleeping.
% ping -c 3 bing                                                                                       @9:54
ping: Unknown host

And from tcpdump I saw this:

# tcpdump -i wlan0 -w file_13.1_NOT_working2.bfp -s 0
tcpdump: listening on wlan0, link-type EN10MB (Ethernet), capture size 262144 bytes
12 packets captured
12 packets received by filter
0 packets dropped by kernel

Of course there is no active network. File is uploaded as file_13.1_NOT_working.bfp.

I also tred /rescue/dhclient wlan0 same result, no network either. File is uploaded as file_13.1_NOT_working2.bfp.

I hope this information helps.
Comment 19 Jaskie 2022-06-03 02:40:15 UTC
Created attachment 234407 [details]
Test file tarball form 13.0 and 13.1

This is a tarball (tar.gz) containing several files.
Comment 20 J.R. Oldroyd 2022-06-03 07:46:45 UTC
(In reply to Jaskie from comment #19)

The dmesg for the 13.1 boot does not show the "[ath_hal] loaded" message.  Was the message removed in 13.1 or does this indicate that the hal is not loaded?
Comment 21 Jaskie 2022-06-03 07:50:59 UTC
(In reply to J.R. Oldroyd from comment #20)

It was directly from dmesg output, I didn't modify anythong. If it's not there then it should mean it was not loaded during bootup. I'll check again tonight when I am around my laptop.
Comment 22 J.R. Oldroyd 2022-06-03 08:02:55 UTC
(In reply to Jaskie from comment #21)

I see in the git log that the ath_hal message was removed and now only shown during a verbose boot.
Comment 23 Jaskie 2022-06-03 09:19:46 UTC
(In reply to J.R. Oldroyd from comment #22)
Do I kldstat to check that or do I have to get dmesg again from a verbose boot?
Comment 24 J.R. Oldroyd 2022-06-03 09:31:41 UTC
(In reply to Jaskie from comment #23)

If you want to verify, you'd have to do a verbose boot.  But you have already told us that the 13.1 kernel works with the 13.0 userland, so not really necessary.
Comment 25 Jaskie 2022-06-03 15:15:17 UTC
I did this steps in both 13.0 and 13.1:

% doas service netif stop wlan0 
% doas mv /var/db/dhclient.leases.wlan0{,.bak}
% doas service netif start wlan0
% doas tcpdump -i wlan0 -w OUT.bpf -s 0
% doas ifconfig wlan0
% doas dhclient wlan0

In 13.0, network connects almost instantly. But in 13.1 it still won't connect. Output similiar to:

DHCPDISCOVER on wlan0 to 255.255.255.255 port 67 interval 4
DHCPDISCOVER on wlan0 to 255.255.255.255 port 67 interval 10
DHCPDISCOVER on wlan0 to 255.255.255.255 port 67 interval 13
DHCPDISCOVER on wlan0 to 255.255.255.255 port 67 interval 9
DHCPDISCOVER on wlan0 to 255.255.255.255 port 67 interval 9
DHCPDISCOVER on wlan0 to 255.255.255.255 port 67 interval 15
No DHCPOFFERS received.
No working leases in persistent database - sleeping.


as before.

I checked rc.conf, wpa_supplicant.conf and they're identical in both BEs. I checked dmesg in both BEs and ath_hal was loaded, didn't know why it was not in the file I uploaded earlier.

To use wireshark or to dig into the tcpducp outputs are beyond my capability so I'll leave it to you as always. Thanks for all your help!

Anyways, this are the tcpdump bpf files: https://tempsend.com/skhrg
Comment 26 Bjoern A. Zeeb freebsd_committer freebsd_triage 2022-06-03 16:32:57 UTC
(In reply to Jaskie from comment #25)

when looking at the first dicover packets between 13.0 and 13.1 I see no obvious differences:

% diff -up dhcp-discover-13.*.txt
--- dhcp-discover-13.0.txt      2022-06-03 16:28:38.732111000 +0000
+++ dhcp-discover-13.1.txt      2022-06-03 16:28:11.892156000 +0000
@@ -1,15 +1,15 @@ No.     Time           Source                Destinati
 No.     Time           Source                Destination           Protocol Length Info
-      3 17.487743      0.0.0.0               255.255.255.255       DHCP     342    DHCP Discover - Transaction ID 0x2770ff8c
+      2 9.494708       0.0.0.0               255.255.255.255       DHCP     342    DHCP Discover - Transaction ID 0xe104ecd1
 
-Frame 3: 342 bytes on wire (2736 bits), 342 bytes captured (2736 bits)
+Frame 2: 342 bytes on wire (2736 bits), 342 bytes captured (2736 bits)
     Encapsulation type: Ethernet (1)
-    Arrival Time: Jun  3, 2022 22:39:04.119281000 UTC
+    Arrival Time: Jun  3, 2022 22:48:38.443883000 UTC
     [Time shift for this packet: 0.000000000 seconds]
-    Epoch Time: 1654295944.119281000 seconds
-    [Time delta from previous captured frame: 17.487210000 seconds]
-    [Time delta from previous displayed frame: 17.487210000 seconds]
-    [Time since reference or first frame: 17.487743000 seconds]
-    Frame Number: 3
+    Epoch Time: 1654296518.443883000 seconds
+    [Time delta from previous captured frame: 9.494708000 seconds]
+    [Time delta from previous displayed frame: 9.494708000 seconds]
+    [Time since reference or first frame: 9.494708000 seconds]
+    Frame Number: 2
     Frame Length: 342 bytes (2736 bits)
     Capture Length: 342 bytes (2736 bits)
     [Frame is marked: True]
@@ -50,7 +50,7 @@ User Datagram Protocol, Src Port: 68, Dst Port: 67
     Source Port: 68
     Destination Port: 67
     Length: 308
-    Checksum: 0x034c [unverified]
+    Checksum: 0x5c72 [unverified]
     [Checksum Status: Unverified]
     [Stream index: 0]
     [Timestamps]
@@ -62,7 +62,7 @@ Dynamic Host Configuration Protocol (Discover)
     Hardware type: Ethernet (0x01)
     Hardware address length: 6
     Hops: 0
-    Transaction ID: 0x2770ff8c
+    Transaction ID: 0xe104ecd1
     Seconds elapsed: 0
     Bootp flags: 0x0000 (Unicast)
         0... .... .... .... = Broadcast flag: Unicast
@@ -104,8 +104,8 @@ Dynamic Host Configuration Protocol (Discover)
 
 0000  ff ff ff ff ff ff 1c 4b d6 ca 3e ac 08 00 45 10   .......K..>...E.
 0010  01 48 00 00 00 00 80 11 39 96 00 00 00 00 ff ff   .H......9.......
-0020  ff ff 00 44 00 43 01 34 03 4c 01 01 06 00 27 70   ...D.C.4.L....'p
-0030  ff 8c 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
+0020  ff ff 00 44 00 43 01 34 5c 72 01 01 06 00 e1 04   ...D.C.4\r......
+0030  ec d1 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
 0040  00 00 00 00 00 00 1c 4b d6 ca 3e ac 00 00 00 00   .......K..>.....
 0050  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
 0060  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................

more in the next reply
Comment 27 Bjoern A. Zeeb freebsd_committer freebsd_triage 2022-06-03 16:40:04 UTC
In the 13.1 not-working case I see incoming TCP packets from the AP despite you not having an IP address (assumed) given DHCP discover did not get a reply.  There's also incoming ARP announcement from the AP-side;  that somehow means the "network" knows where to find you (with your old IP address)?

Given I have no insight in how complex this network is (is it just a residential router with builtin WiFi or more components) it is hard to say where to start looking.

You should probably wait at least an ARP expiry timeout before running the 2nd test.

The other interesting bit would be:

ifconfig wlan0 inet6 -ifdisabled accept_rtadv
ping6 -n ff02::1%wlan0
ping6 -n ff02::2%wlan0

and see if you get replies in the non-working IPv4 case given there seems to be at least IPv6 link-local available on the RuiJieNe node.
Comment 28 J.R. Oldroyd 2022-06-03 18:52:23 UTC
The ARPs from the router are almost certainly due to the previous good 13.0 boot.

I also see (using tcpdump and tshark) that the DHCPDISCOVER datagrams are the same between the 13.0 and 13.1 files.

If you send a server the same request, it should respond the same way.  The fact that, on 13.1, no response is received, has to indicate that the outbound DISCOVER was blocked and so the DHCP server didn't receive the request, or that it did, but the response OFFER was blocked.

I suggested in email that it's firewall or something on the host that is different between 13.0 and 13.1 and blocking the traffic.  But Jackie confirms that the rc.conf is the same.

A DHCP server will (should) respond to repeated requests from the same host even before the lease renewal time.  You can test this by booting the working 13.0, obtaining the DHCP lease, killing dhclient, removing the lease file and starting dhclient again.  You'll get a new lease immediately for sure.  (But, if you don't, that would point at the problem.)

ARP problem?  Unlikely I think, because it is the same host computer using the same MAC address talking to the same server.  You can use "arp -d" to clear ARP entries on the host.  If the ARP entry on the router is the problem, you'll have to wait.  Cisco routers use a 4 hour cache time, I believe.  Configure it for the 13.1 env, turn off, wait.  After the time, boot straight into 13.1 env without booting 13.0 first.
Comment 29 J.R. Oldroyd 2022-06-03 19:01:16 UTC
The other obvious thing to check is, is the network working when configured manually?

Use the same values you get from the 13.0 boot for the IP and router.  Then boot under 13.1 and configure manually:

    ifconfig wlan0 10.134.138.51 netmask 255.255.128.0
    route add default 10.134.255.254

Can you ping out?  Can you ping the router?  Can you ping anything beyond the router?
Comment 30 Jaskie 2022-06-04 01:36:44 UTC
I followed all your suggestions and did the tests below with a fresh biot 13.1, dhclient cache was removed beforehand:

# ifconfig wlan0 10.134.138.51 netmask 255.255.128.0 
# route add default 10.134.255.254
# ping bing.com                                                                                        
ping: Unknown host
# ifconfig wlan0 inet6 -ifdisabled accept_rtadv                                                        
# ping6 -n ff02::1%wlan0
PING6(56=40+8+8 bytes) fe80::1e4b:d6ff:feca:3eac%wlan0 --> ff02::1%wlan0                                                                                                                   
ping6: sendmsg: No buffer space available                                                                                                                                                  
ping6: wrote ff02::1%wlan0 16 chars, ret=-1                                                                                                                                                
16 bytes from fe80::1e4b:d6ff:feca:3eac%wlan0, icmp_seq=0 hlim=64 time=0.162 ms                                                                                                            
ping6: sendmsg: No buffer space available                                                                                                                                                  
ping6: wrote ff02::1%wlan0 16 chars, ret=-1                                                                                                                                                
16 bytes from fe80::1e4b:d6ff:feca:3eac%wlan0, icmp_seq=1 hlim=64 time=0.074 ms                                                                                                            
ping6: sendmsg: No buffer space available                                                                                                                                                  
ping6: wrote ff02::1%wlan0 16 chars, ret=-1                                                                                                                                                
16 bytes from fe80::1e4b:d6ff:feca:3eac%wlan0, icmp_seq=2 hlim=64 time=0.104 ms                                                                                                            
ping6: sendmsg: No buffer space available                                                                                                                                                  
ping6: wrote ff02::1%wlan0 16 chars, ret=-1                                                                                                                                                
16 bytes from fe80::1e4b:d6ff:feca:3eac%wlan0, icmp_seq=3 hlim=64 time=0.097 ms                                                                                                            
ping6: sendmsg: No buffer space available                                                                                                                                                  
ping6: wrote ff02::1%wlan0 16 chars, ret=-1                                                                                                                                                
16 bytes from fe80::1e4b:d6ff:feca:3eac%wlan0, icmp_seq=4 hlim=64 time=0.100 ms                                                                                                            
ping6: sendmsg: No buffer space available                                                                                                                                                  
ping6: wrote ff02::1%wlan0 16 chars, ret=-1                                                                                                                                                
16 bytes from fe80::1e4b:d6ff:feca:3eac%wlan0, icmp_seq=5 hlim=64 time=0.076 ms                                                                                                            
^C                                                                                                                                                                                         
--- ff02::1%wlan0 ping6 statistics ---                                                                                                                                                     
6 packets transmitted, 6 packets received, 0.0% packet loss                                                                                                                                
round-trip min/avg/max/std-dev = 0.074/0.102/0.162/0.029 ms  

# ping6 -n ff02::2%wlan0                                                                               
PING6(56=40+8+8 bytes) fe80::1e4b:d6ff:feca:3eac%wlan0 --> ff02::2%wlan0                                                     
ping6: sendmsg: No buffer space available                                                                                    
ping6: wrote ff02::2%wlan0 16 chars, ret=-1                                                                                  
ping6: sendmsg: No buffer space available                                                                                    
ping6: wrote ff02::2%wlan0 16 chars, ret=-1                                                                                  
ping6: sendmsg: No buffer space available                                                                                    
ping6: wrote ff02::2%wlan0 16 chars, ret=-1                                                                                  
ping6: sendmsg: No buffer space available                                                                                    
ping6: wrote ff02::2%wlan0 16 chars, ret=-1                                                                                  
ping6: sendmsg: No buffer space available                                                                                    
ping6: wrote ff02::2%wlan0 16 chars, ret=-1                                                                                  
ping6: sendmsg: No buffer space available                                                                                    
ping6: wrote ff02::2%wlan0 16 chars, ret=-1                                                                                  
^C                                                                                                                           
--- ff02::2%wlan0 ping6 statistics ---                                                                                       
6 packets transmitted, 0 packets received, 100.0% packet loss 

# ping -c 3 10.134.138.51                                                                              @9:07
PING 10.134.138.51 (10.134.138.51): 56 data bytes
64 bytes from 10.134.138.51: icmp_seq=0 ttl=64 time=0.058 ms
64 bytes from 10.134.138.51: icmp_seq=1 ttl=64 time=0.039 ms
64 bytes from 10.134.138.51: icmp_seq=2 ttl=64 time=0.047 ms

--- 10.134.138.51 ping statistics ---
3 packets transmitted, 3 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 0.039/0.048/0.058/0.008 ms


# ping -c 3 255.255.128.0                                                                              @9:07
PING 255.255.128.0 (255.255.128.0): 56 data bytes
ping: sendto: No buffer space available
ping: sendto: No buffer space available
ping: sendto: No buffer space available
^C
--- 255.255.128.0 ping statistics ---
3 packets transmitted, 0 packets received, 100.0% packet loss

# ping -c 3 10.134.255.254                                                                             @9:07
PING 10.134.255.254 (10.134.255.254): 56 data bytes
ping: sendto: No buffer space available
ping: sendto: No buffer space available
ping: sendto: No buffer space available

--- 10.134.255.254 ping statistics ---
3 packets transmitted, 0 packets received, 100.0% packet loss

# ping 1.1.1.1                                                                                                                                                       @9:21
PING 1.1.1.1 (1.1.1.1): 56 data bytes
ping: sendto: No buffer space available
ping: sendto: No buffer space available
ping: sendto: No buffer space available
ping: sendto: No buffer space available
^C
--- 1.1.1.1 ping statistics ---
4 packets transmitted, 0 packets received, 100.0% packet loss

# ifconfig wlan0
wlan0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        ether 1c:4b:d6:ca:3e:ac
        inet 10.134.138.51 netmask 0xffff8000 broadcast 10.134.255.255
        inet6 fe80::1e4b:d6ff:feca:3eac%wlan0 prefixlen 64 scopeid 0x3
        groups: wlan
        ssid WHU-STU channel 5 (2432 MHz 11g ht/20) bssid 70:d9:31:0e:2c:00
        regdomain 96 indoor ecm authmode WPA privacy MIXED deftxkey UNDEF
        txpower 20 bmiss 7 scanvalid 60 protmode CTS ampdulimit 64k shortgi
        -stbctx stbcrx -ldpc -uapsd wme burst roaming MANUAL bintval 300
        parent interface: ath0
        media: IEEE 802.11 Wireless Ethernet DS/1Mbps mode 11ng
        status: associated
        nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL>

I don't know if I am doing this the right way. I am now leaving my laptop on after arp -d -i wlan0 -a to see what happens tonight.

I almost thought network was back when I saw networkmgr on my system tray icon changed to indicate network was up and ifconfig also reported a valid IP, but obviously nothing really changed.
Comment 31 J.R. Oldroyd 2022-06-04 09:19:05 UTC
(In reply to Jaskie from comment #30)

I notice you're associated again with the bssid which requires WPA auth:
    ssid WHU-STU channel 5 (2432 MHz 11g ht/20) bssid 70:d9:31:0e:2c:00
    regdomain 96 indoor ecm authmode WPA privacy MIXED deftxkey UNDEF


In your working 13.0 comments you have been on a bssid which is OPEN and you have said you go to a sign-in page to authenticate:
    ssid WHU-STU channel 13 (2472 MHz 11g ht/20) bssid fc:b6:98:f6:12:80
    regdomain 96 indoor ecm authmode OPEN privacy OFF txpower 20 bmiss 7

As I suggested in email on May 29th, have you tried forcing the use of the known working bssid by adding it to the network details in wpa_supplicant.conf:
	network={
		ssid="WHU-STU"
		bssid=fc:b6:98:f6:12:80
		key_mgmt=NONE
	}

You must remove (comment out) all other WHU-STU network config so that the only bssid it associates with is the one that is known to work.  When you run ifconfig, it must show the known good bssid.
Comment 32 Jaskie 2022-06-04 12:57:44 UTC
(In reply to J.R. Oldroyd from comment #31)

I would not say this is a problem. As I mentioned in my last reply, /etc/rc.conf, /etc/wpa_supplicant.conf and /etc/sysctl.conf are identical between 13.0 and 13.1. And for clarification, there is only one network named WHU-STU (no passwork required to connect, but do require login via a web page). It works in 13.0 regardless of whether or not I assign a bssid for it. Well then, if it works in 13.0, shouldn't it, with exactly the same config files, also just work in 13.1?

If I remember it correctly, I now have 

	network={
		ssid="WHU-STU"
		bssid=70:d9:31:0e:2c:00
		key_mgmt=NONE
	}

in 13.1 and 13.0. It works in 13.0 but not in 13.1. Meanwhile, I'll check with the other bssid later and get back to you. As always, Thanks for your help along the way!
Comment 33 Bjoern A. Zeeb freebsd_committer freebsd_triage 2022-06-04 14:34:39 UTC
(In reply to Jaskie from comment #32)

After failure check the last 100 lines (or whatever is reasonable based on timestamp) of the ls -lart /var/log logfiles   messages probably mostly;  see if there are any errors or warnings ... is there anything from wpa_supplicant?
Comment 34 Jaskie 2022-06-04 14:46:29 UTC
(In reply to Jaskie from comment #32)

Network is still down if I omit bssid or assign bssid=fc:b6:98:f6:12:80 for it.

On the other hand, the laptop was left on all day after arp -d and still nothing. I think this rule out the arp cache thing?
Comment 35 Jaskie 2022-06-04 15:44:58 UTC
(In reply to Bjoern A. Zeeb from comment #33)

I assigned bssid=fc:b6:98:f6:12:80 in /etc/wpa_supplicant.conf again, made sure it worked in 13.0. And below are terminal inputs (% as PS1) and outputs in 13.1:

% doas service netif restart                                                                                                                                                                             
dhclient not running? (check /var/run/dhclient/dhclient.alc0.pid).                                                                                                                         
Stopping wpa_supplicant.                                                                                                                                                                   
Waiting for PIDS: 333.                                                                                                                                                                     
Stopping Network: lo0 alc0 wlan0.                                                                                                                                                          
lo0: flags=8048<LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384                                                                                                                             
        options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>                                                                                                                    
        groups: lo                                                                                                                                                                         
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>                                                                                                                                          
alc0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500                                                                                                                            
        options=c3198<VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_MCAST,WOL_MAGIC,VLAN_HWTSO,LINKSTATE>                                                                                   
        ether 48:5b:39:8e:3c:b0                                                                                                                                                            
        media: Ethernet autoselect                                                                                                                                                         
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>                                                                                                                               
wlan0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500                                                                                                                           
        ether 1c:4b:d6:ca:3e:ac                                                                                                                                                            
        groups: wlan                                                                                                                                                                       
        ssid "" channel 13 (2472 MHz 11g ht/20)                                                                                                                                            
        regdomain 96 indoor ecm authmode OPEN privacy OFF txpower 20 bmiss 7                                                                                                               
        scanvalid 60 protmode CTS ampdulimit 64k ampdudensity 8 shortgi                                                                                                                    
        -stbctx stbcrx -ldpc -uapsd wme burst                                                                                                                                              
        parent interface: ath0                                                                                                                                                             
        media: IEEE 802.11 Wireless Ethernet autoselect (autoselect)                                                                                                                       
        status: no carrier                                                                                                                                                                 
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>                                                                                                                               
Destroyed wlan(4) interfaces: wlan0.                                                                                                                                                       
Created wlan(4) interfaces: wlan0.                                                                                                                                                         
Starting wpa_supplicant.                                                                                                                                                                   
Starting Network: lo0 alc0 wlan0.                                                                                                                                                          
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384                                                                                                                          
        options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>                                                                                                                    
        inet6 ::1 prefixlen 128                                                                                                                                                            
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2                                                                                                                                         
        inet 127.0.0.1 netmask 0xff000000                                                                                                                                                  
        groups: lo                                                                                                                                                                         
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>                                                                                                                                          
alc0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500                                                                                                                 
        options=c3198<VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_MCAST,WOL_MAGIC,VLAN_HWTSO,LINKSTATE>                                                                                   
        ether 48:5b:39:8e:3c:b0                                                                                                                                                            
        media: Ethernet autoselect (none)                                                                                                                                                  
        status: no carrier                                                                                                                                                                 
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>                                                                                                                               
wlan0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500                                                                                                                
        ether 1c:4b:d6:ca:3e:ac
        groups: wlan                          
        ssid "" channel 13 (2472 MHz 11g ht/20)                                              
        regdomain 96 indoor ecm authmode WPA1+WPA2/802.11i privacy MIXED                                                                                                                   
        deftxkey UNDEF txpower 20 bmiss 7 scanvalid 60 protmode CTS                                                                                                                        
        ampdulimit 64k ampdudensity 8 shortgi -stbctx stbcrx -ldpc -uapsd wme                                                                                                              
        burst roaming MANUAL bintval 0                                                       
        parent interface: ath0                
        media: IEEE 802.11 Wireless Ethernet autoselect (autoselect)                                                                                                                       
        status: no carrier                    
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>

% doas dhclient wlan0                                                                                                                                               @23:24
wlan0: no link ........ got link
DHCPDISCOVER on wlan0 to 255.255.255.255 port 67 interval 4
DHCPDISCOVER on wlan0 to 255.255.255.255 port 67 interval 4
DHCPDISCOVER on wlan0 to 255.255.255.255 port 67 interval 8
DHCPDISCOVER on wlan0 to 255.255.255.255 port 67 interval 17
DHCPDISCOVER on wlan0 to 255.255.255.255 port 67 interval 17
DHCPDISCOVER on wlan0 to 255.255.255.255 port 67 interval 5
No DHCPOFFERS received.
No working leases in persistent database - sleeping.

% ifconfig wlan0                                                                                                                                                    @23:27
wlan0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        ether 1c:4b:d6:ca:3e:ac
        inet 0.0.0.0/8 broadcast 255.255.255.255
        groups: wlan
        ssid WHU-STU channel 13 (2472 MHz 11g ht/20) bssid fc:b6:98:f6:12:80
        regdomain 96 indoor ecm authmode WPA privacy MIXED deftxkey UNDEF
        txpower 20 bmiss 7 scanvalid 60 protmode CTS ampdulimit 64k shortgi
        -stbctx stbcrx -ldpc -uapsd wme burst roaming MANUAL bintval 300
        parent interface: ath0
        media: IEEE 802.11 Wireless Ethernet DS/1Mbps mode 11ng
        status: associated
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>

Based on the time I executed those commands, here are log file entries I saw:

In /var/log/messages:

Jun  4 23:24:12 freebsd wpa_supplicant[2383]: wlan0: Trying to associate with fc:b6:98:f6:12:80 (SSID='WHU-STU' freq=2472 MHz)
Jun  4 23:24:12 freebsd wpa_supplicant[2383]: wlan0: Associated with fc:b6:98:f6:12:80
Jun  4 23:24:12 freebsd kernel: wlan0: link state changed to UP
Jun  4 23:24:12 freebsd wpa_supplicant[2383]: wlan0: CTRL-EVENT-CONNECTED - Connection to fc:b6:98:f6:12:80 completed [id=1 id_str=]
Jun  4 23:24:40 freebsd kernel: lo0: link state changed to DOWN
Jun  4 23:24:40 freebsd wpa_supplicant[2383]: wlan0: CTRL-EVENT-DISCONNECTED bssid=fc:b6:98:f6:12:80 reason=3 locally_generated=1
Jun  4 23:24:40 freebsd kernel: wlan0: link state changed to DOWN
Jun  4 23:24:40 freebsd wpa_supplicant[2383]: wlan0: CTRL-EVENT-DSCP-POLICY clear_all
Jun  4 23:24:40 freebsd syslogd: last message repeated 1 times
Jun  4 23:24:40 freebsd wpa_supplicant[2383]: wlan0: CTRL-EVENT-TERMINATING 
Jun  4 23:24:41 freebsd kernel: wlan0: Ethernet address: 1c:4b:d6:ca:3e:ac
Jun  4 23:24:41 freebsd kernel: lo0: link state changed to UP
Jun  4 23:24:41 freebsd wpa_supplicant[3069]: Successfully initialized wpa_supplicant
Jun  4 23:24:41 freebsd wpa_supplicant[3069]: ioctl[SIOCS80211, op=20, val=0, arg_len=7]: Invalid argument
Jun  4 23:24:41 freebsd syslogd: last message repeated 1 times
Jun  4 23:24:41 freebsd wpa_supplicant[3071]: wlan0: CTRL-EVENT-SCAN-FAILED ret=-1 retry=1
Jun  4 23:24:42 freebsd syslogd: last message repeated 1 times
Jun  4 23:24:59 freebsd wpa_supplicant[3071]: wlan0: Trying to associate with fc:b6:98:f6:12:80 (SSID='WHU-STU' freq=2472 MHz)
Jun  4 23:24:59 freebsd wpa_supplicant[3071]: wlan0: Associated with fc:b6:98:f6:12:80
Jun  4 23:24:59 freebsd kernel: wlan0: link state changed to UP
Jun  4 23:24:59 freebsd wpa_supplicant[3071]: wlan0: CTRL-EVENT-CONNECTED - Connection to fc:b6:98:f6:12:80 completed [id=1 id_str=]
Jun  4 23:24:59 freebsd dhclient[3136]: send_packet: No buffer space available
Jun  4 23:25:32 freebsd syslogd: last message repeated 4 times
Jun  4 23:25:49 freebsd syslogd: last message repeated 1 times


In /var/log/daemon.log:

Jun  4 23:24:12 freebsd wpa_supplicant[2383]: wlan0: Trying to associate with fc:b6:98:f6:12:80 (SSID='WHU-STU' freq=2472 MHz)
Jun  4 23:24:12 freebsd wpa_supplicant[2383]: wlan0: Associated with fc:b6:98:f6:12:80
Jun  4 23:24:12 freebsd wpa_supplicant[2383]: wlan0: CTRL-EVENT-CONNECTED - Connection to fc:b6:98:f6:12:80 completed [id=1 id_str=]
Jun  4 23:24:40 freebsd wpa_supplicant[2383]: wlan0: CTRL-EVENT-DISCONNECTED bssid=fc:b6:98:f6:12:80 reason=3 locally_generated=1
Jun  4 23:24:40 freebsd wpa_supplicant[2383]: wlan0: CTRL-EVENT-DSCP-POLICY clear_all
Jun  4 23:24:40 freebsd syslogd: last message repeated 1 times
Jun  4 23:24:40 freebsd wpa_supplicant[2383]: wlan0: CTRL-EVENT-TERMINATING 
Jun  4 23:24:41 freebsd wpa_supplicant[3069]: Successfully initialized wpa_supplicant
Jun  4 23:24:41 freebsd wpa_supplicant[3069]: ioctl[SIOCS80211, op=20, val=0, arg_len=7]: Invalid argument
Jun  4 23:24:41 freebsd syslogd: last message repeated 1 times
Jun  4 23:24:41 freebsd wpa_supplicant[3071]: wlan0: CTRL-EVENT-SCAN-FAILED ret=-1 retry=1
Jun  4 23:24:42 freebsd syslogd: last message repeated 1 times
Jun  4 23:24:59 freebsd wpa_supplicant[3071]: wlan0: Trying to associate with fc:b6:98:f6:12:80 (SSID='WHU-STU' freq=2472 MHz)
Jun  4 23:24:59 freebsd wpa_supplicant[3071]: wlan0: Associated with fc:b6:98:f6:12:80
Jun  4 23:24:59 freebsd wpa_supplicant[3071]: wlan0: CTRL-EVENT-CONNECTED - Connection to fc:b6:98:f6:12:80 completed [id=1 id_str=]
Jun  4 23:24:59 freebsd dhclient[3136]: DHCPDISCOVER on wlan0 to 255.255.255.255 port 67 interval 4
Jun  4 23:24:59 freebsd dhclient[3136]: send_packet: No buffer space available
Jun  4 23:25:03 freebsd dhclient[3136]: DHCPDISCOVER on wlan0 to 255.255.255.255 port 67 interval 4
Jun  4 23:25:03 freebsd dhclient[3136]: send_packet: No buffer space available
Jun  4 23:25:07 freebsd dhclient[3136]: DHCPDISCOVER on wlan0 to 255.255.255.255 port 67 interval 8
Jun  4 23:25:07 freebsd dhclient[3136]: send_packet: No buffer space available
Jun  4 23:25:15 freebsd dhclient[3136]: DHCPDISCOVER on wlan0 to 255.255.255.255 port 67 interval 17
Jun  4 23:25:15 freebsd dhclient[3136]: send_packet: No buffer space available
Jun  4 23:25:32 freebsd dhclient[3136]: DHCPDISCOVER on wlan0 to 255.255.255.255 port 67 interval 17
Jun  4 23:25:32 freebsd dhclient[3136]: send_packet: No buffer space available
Jun  4 23:25:49 freebsd dhclient[3136]: DHCPDISCOVER on wlan0 to 255.255.255.255 port 67 interval 5
Jun  4 23:25:49 freebsd dhclient[3136]: send_packet: No buffer space available
Jun  4 23:25:54 freebsd dhclient[3136]: No DHCPOFFERS received.
Jun  4 23:25:54 freebsd dhclient[3136]: No working leases in persistent database - sleeping.

And in /var/log/dmesg.today (just last lines seemed relavent here, not sure about time though 'cuz there was no timestamp):

wlan0: link state changed to UP
uhub1: 6 ports with 6 removable, self powered
uhub0: 6 ports with 6 removable, self powered
ugen7.2: <Sonix Technology Co., Ltd. USB 2.0 Camera> at usbus7
ath0: bb hang detected (0x4), resetting
ath0: bb hang detected (0x4), resetting
ath0: bb hang detected (0x4), resetting
ath0: bb hang detected (0x4), resetting
ath0: bb hang detected (0x4), resetting
Comment 36 Jaskie 2022-06-05 14:26:50 UTC
I followed suggestion from J.R. Oldroyd and copied /usr/sbin/wpa_supplicant from 13.0-p11 to 13.1 and network is finally working! Should this bug report be re-assigned?
Comment 37 rkoberman 2022-06-05 20:29:18 UTC
I am surprised to see this issue as the initial message showed both 13.1 and 13.0 to be associated with an SSID "WHU-STU". I should have looked more closely as I missed that they are the same SSID with radically different configurations. Both come from the same vendor but, with 13.1, both are setting up in a broken manner. "media: IEEE 802.11 Wireless Ethernet DS/1Mbps mode 11ng" is probably nonsensical. At least it is certainly wrong! DS is obsolete and has not  ben use in year. 1Mbps does make some sense if your are using DS, but, even then, it's weird.

I wish I could look at the ifconfig from when I was at the hotel, but I have not recorded it anywhere.

The results of this WPA configured mode is very unlikely to work. I'm guessing that the issue is that the APs at your location is using MCS. Bet the hotel is, too. It's relatively new and I suspect the 13.1 wpa_supplicant is broken in how it deals with it. My home router does not use it.

Only about a half dozen commits, from a quick look, to wpa_supplicant, so bisecting should not take too long, especially since it is in user space. If I  had access to an AP that used MCS for selection, I'd do it myself, but I don't. If you have the interest and the time, it will be fairly simple. (One of the big things you get with git). teh commits of interest are:
wpa: Import wpa 2.10.	Cy Schubert	2022-02-08	22	-1500/+98
wpa: Import wpa_supplicant/hostapd commit b26f5c0fe	Cy Schubert	2022-02-08	27	-110/+824
wpa: Import wpa_supplicant/hostapd commit 14ab4a816	Cy Schubert	2022-01-03	33	-203/+3129
wpa: Import wpa_supplicant/hostapd commits up to b4f7506ff	Cy Schubert	2021-11-04	105	-2474/+19727
wpa: Add wpa_cli action file event	Cy Schubert	2021-08-20	1	-0/+2
Comment 38 rkoberman 2022-06-05 20:37:04 UTC
BTW, this appears to have nothing specifically to do with the AR9285. It is likely fundamental to the wpa supplicant when dealing with a some modern APs. In my case, I was seeing it on an Intel AX200.
Comment 39 Cy Schubert freebsd_committer freebsd_triage 2022-06-06 01:33:52 UTC
Created attachment 234478 [details]
Correctly call pcap_next_ex

This patch failed to make it into 13.1-RELEASE and fixes the problem.
Comment 40 Cy Schubert freebsd_committer freebsd_triage 2022-06-06 01:43:36 UTC
I suggest we add the patch to releng/13.1 and publish an errata.
Comment 41 Cy Schubert freebsd_committer freebsd_triage 2022-06-06 01:52:49 UTC
Assigning to releng@. The patch that fixes this is in main and stable/13. MFS d7365cc80c6532973e529195bddf9dbf7f35c239 from stable/13 to releng/13.1 is recommended.
Comment 42 Jaskie 2022-06-06 15:18:03 UTC
Unfortunately the fixntou mentioned doesn't seem to solve my issue. I followed J.R. Oldroyd's guide and did the following test by compiling WPA:


cd /usr/src
git clone https://git.freebsd.org/src.git .
git switch releng/13.1


git cherry-pick 1e0ca65a3bb5798a80eccaae58d863f1f08b9ae8
git commit -m "msg"

cd usr.sbin/wpa
make
mv /usr/sbin/wpa_supplicant{,.bak 13.0OK}

cp -p /usr/obj/usr/src/amd64.amd64/usr.sbin/wpa/wpa_supplicant/wpa_supplicant /usr/sbin/wpa_supplicant
strip /usr/sbin/wpa_supplicant

reboot




Ln reboot, it's all the same as I reported in the first message of this bug report, nothing really changed as I noticed. But as soon as I copied wpa_supplicant 13.0 in place and restart netif, everything just works again.
Comment 43 Cy Schubert freebsd_committer freebsd_triage 2022-06-06 16:01:07 UTC
Created attachment 234491 [details]
Two patches in current and stable/13 that are not in 13.1

This problem is similar if not the same as a problem reported against 14-CURRENT in April. Unfortunately, both patches are not in 13.1. Can you try both, please.
Comment 44 Jaskie 2022-06-09 05:14:50 UTC
(In reply to Cy Schubert from comment #43)

I am sorry I am having some trouble testing the two patches. I am not very familiar with the process of applying patches from different branches. However I followed guidance by J.R. Oldroyd and tried git bisecct.

wpa_supplicant in 13.0 (which is known to work for me) has a date shown by ls -l as 2021-10-02

And

cd /usr/src
git reset --hard release/13.1.0
cd /usr/src/contrib/wpa
git log .

showed these commits in order:

094f618ca12e42243a6ebcccde3c3f616645e7ba 2022/02/08                                          
f3ba7fe0d2202c9a17513982ed5ac767618afbdc 2022/02/08                                         
9d04424634024aef04bafa3104f24e0df9f4cce7 2022/01/03                                         
0a6760a1de32bf5df91ef926eba25b3f74b4f84f 2021/11/04
e3928ece86ca9cae6734278d4e40673b21fed417 2021/09/04                                                       

So the 13.0 working wpa happened just after e3928ece86ca9cae6734278d4e40673b21fed417, which means that something broke wpa between e3928ece86ca9cae6734278d4e40673b21fed417 (2021/09/04) and 094f618ca12e42243a6ebcccde3c3f616645e7ba (2022/02/08). Next I started to git bisect by:

cd /usr/src
git bisect start 094f618ca12e42243a6ebcccde3c3f616645e7ba e3928ece86ca9cae6734278d4e40673b21fed417 -- contrib/wpa

Then there were two steps tests, which I then went through only to fiind out both failed so at the end I got this:

git bisect bad
0a6760a1de32bf5df91ef926eba25b3f74b4f84f is the first bad commit
commit 0a6760a1de32bf5df91ef926eba25b3f74b4f84f
Author: Cy Schubert <cy@FreeBSD.org>
Date: Fri Sep 3 06:07:19 2021 -0700

    wpa: Import wpa_supplicant/hostapd commits up to b4f7506ff
     
    Merge vendor commits 40c7ff83e74eabba5a7e2caefeea12372b2d3f9a,
    efec8223892b3e677acb46eae84ec3534989971f, and
    2f6c3ea9600b494d24cac5a38c1cea0ac192245e.
     
    Tested by: philip
     
    (cherry picked from commit c1d255d3ffdbe447de3ab875bf4e7d7accc5bfc5)



To double check I also did:

cd /usr/src
git checkout e3928ece86ca9cae6734278d4e40673b21fed417
cd /usr/src/usr.sbin/wpa
make clean
make

and the resulting wpa_supplicant worked as expected.
Hope this information helps, thanks.
Comment 45 Cy Schubert freebsd_committer freebsd_triage 2022-06-09 05:48:26 UTC
(In reply to Jaskie from comment #44)
Excellent! Thank you for verifying this.

I'll ask re@ (by assigning the PR to re@) if they can merge the two patches from stable/13 into releng/13.1, and publish an errata. This patch didn't quite make it into releng/13.1.

The two patches fix a multitude of problems in wpa_supplicant and hostapd because a legitimate zero bytes returned can be confused with an error resulting in zero bytes returned. pcap_next_ex() solves this for us by returning the return code separately from the bytes returned whereas pcap_next() results in a confusing result.

The other alternative would be to use the security/wpa_supplicant port. It includes the patch.

BTW, you should only need to do a git apply against the patch. Then cd /usr/src/usr.sbin/wpa. Then make obj; make depend; make includes; make; make install.

re@, would it be possible to publish an errata for this?
Comment 46 Jaskie 2022-06-09 10:26:32 UTC
(In reply to Cy Schubert from comment #45)

Do I need only one patch? I thought I needed two, but I didn't figure out where t find them and how to apply and verify the expected change.
Comment 47 Jaskie 2022-06-09 14:38:10 UTC
(In reply to Cy Schubert from comment #45)

So I finally tested the patch you mentioned with the help of J.R. Oldroyd by git cherry-picking. Well, it didn't work either. This is what I did:

cd /usr/src/
git reset --hard release/13.1.0
git cherry-pick 6e5d01124fd4dd57899ddd9260c76dbb43543aa7
git cherry-pick 1e0ca65a3bb5798a80eccaae58d863f1f08b9ae8
git diff 094f618ca12e contrib/wpa
git diff 094f618ca12e contrib/wpa > /home/adam/diff.log


Then I did diff /home/adam/{diff.log,patch.txt} and it showed (patch.txt was download from this PR, the file you uploaded: https://bugs.freebsd.org/bugzilla/attachment.cgi?id=234491):

2c2
< index 48e18fffba5..0461758ff21 100644
---
> index 48e18fffba57..0461758ff210 100644

I thought this confirmed the two cherrypicks did the same patch?

However, I then went on compile usr.sbin/wpa and tested resulting wpa_supplicant, but it still didn't work.
Comment 48 Cy Schubert freebsd_committer freebsd_triage 2022-06-09 15:08:31 UTC
(In reply to Jaskie from comment #35)
Looking at these messages, wpa_supplicant has associated with the AP.

Can you try an experiment, please.

When this condition occurs,

ifconfig wlan0 -- and record the output
killall dhclient
ifconfig wlan0 down
wait 30 seconds
ifconfig wlan0 up
ifconfig wlan0 -- record the output
dhclient wlan0
Wait for it to get an ip, then run ifconfig wlan0 to record the output.

Also egrep -i 'wpa_supplicant|dhclient' /var/log/messages
Comment 49 Jaskie 2022-06-09 15:34:21 UTC
(In reply to Cy Schubert from comment #48)

This is all the output:


# doas service netif start wlan0
# ifocnfig wlan0
wlan0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        ether 1c:4b:d6:ca:3e:ac
        inet 0.0.0.0/8 broadcast 255.255.255.255
        groups: wlan
        ssid WHU-STU channel 5 (2432 MHz 11g ht/20) bssid 70:d9:31:0e:2c:00
        regdomain 96 indoor ecm authmode WPA privacy MIXED deftxkey UNDEF
        txpower 20 bmiss 7 scanvalid 60 protmode CTS ampdulimit 64k shortgi
        -stbctx stbcrx -ldpc -uapsd wme burst roaming MANUAL bintval 300
        parent interface: ath0
        media: IEEE 802.11 Wireless Ethernet DS/1Mbps mode 11ng
        status: associated
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
# doas ifconfig wlan0 down
# sleep 30
# doas ifconfig wlan0 up
# ifconfig wlan0
wlan0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        ether 1c:4b:d6:ca:3e:ac
        inet 0.0.0.0/8 broadcast 255.255.255.255
        groups: wlan
        ssid WHU-STU channel 5 (2432 MHz 11g ht/20) bssid 70:d9:31:0e:2c:00
        regdomain 96 indoor ecm authmode WPA privacy MIXED deftxkey UNDEF
        txpower 20 bmiss 7 scanvalid 60 protmode CTS ampdulimit 64k shortgi
        -stbctx stbcrx -ldpc -uapsd wme burst roaming MANUAL bintval 300
        parent interface: ath0
        media: IEEE 802.11 Wireless Ethernet DS/1Mbps mode 11ng
        status: associated
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
# doas dhclient wlan0                                                                         
dhclient already running, pid: 32840.
exiting.
# doas killall dhclient                                                                       # doas dhclient wlan0                                                                         
DHCPREQUEST on wlan0 to 255.255.255.255 port 67
DHCPREQUEST on wlan0 to 255.255.255.255 port 67
DHCPREQUEST on wlan0 to 255.255.255.255 port 67
DHCPDISCOVER on wlan0 to 255.255.255.255 port 67 interval 7
DHCPDISCOVER on wlan0 to 255.255.255.255 port 67 interval 12
DHCPDISCOVER on wlan0 to 255.255.255.255 port 67 interval 17
DHCPDISCOVER on wlan0 to 255.255.255.255 port 67 interval 10
DHCPDISCOVER on wlan0 to 255.255.255.255 port 67 interval 15
No DHCPOFFERS received.
Trying recorded lease 10.134.135.174
No working leases in persistent database - sleeping.

au the end, because wpa is not working so I won't get an IP at all:

# ifconfig wlan0 
wlan0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        ether 1c:4b:d6:ca:3e:ac
        groups: wlan
        ssid WHU-STU channel 5 (2432 MHz 11g ht/20) bssid 70:d9:31:0e:2c:00
        regdomain 96 indoor ecm authmode WPA privacy MIXED deftxkey UNDEF
        txpower 20 bmiss 7 scanvalid 60 protmode CTS ampdulimit 64k shortgi
        -stbctx stbcrx -ldpc -uapsd wme burst roaming MANUAL bintval 300
        parent interface: ath0
        media: IEEE 802.11 Wireless Ethernet DS/1Mbps mode 11ng
        status: associated
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>


# egrep -i 'wpa_supplicant|dhclient' /var/log/messages

Jun  9 23:18:36 freebsd wpa_supplicant[32528]: Successfully initialized wpa_supplicant
Jun  9 23:18:36 freebsd wpa_supplicant[32528]: ioctl[SIOCS80211, op=20, val=0, arg_len=7]: Invalid argument
Jun  9 23:18:36 freebsd wpa_supplicant[32529]: wlan0: CTRL-EVENT-SCAN-FAILED ret=-1 retry=1
Jun  9 23:18:39 freebsd wpa_supplicant[32529]: wlan0: Trying to associate with 70:d9:31:0e:2c:00 (SSID='WHU-STU' freq=2432 MHz)
Jun  9 23:18:39 freebsd wpa_supplicant[32529]: wlan0: Associated with 70:d9:31:0e:2c:00
Jun  9 23:18:39 freebsd wpa_supplicant[32529]: wlan0: CTRL-EVENT-CONNECTED - Connection to 70:d9:31:0e:2c:00 completed [id=1 id_str=]
Jun  9 23:18:40 freebsd dhclient[32580]: send_packet: No buffer space available
Jun  9 23:19:55 freebsd dhclient[32660]: New IP Address (wlan0): 10.134.135.174
Jun  9 23:19:55 freebsd dhclient[32664]: New Subnet Mask (wlan0): 255.255.128.0
Jun  9 23:19:55 freebsd dhclient[32668]: New Broadcast Address (wlan0): 10.134.255.255
Jun  9 23:19:55 freebsd dhclient[32672]: New Routers (wlan0): 10.134.255.254
Jun  9 23:19:56 freebsd dhclient[32677]: New Routers (wlan0): 10.134.255.254
Jun  9 23:20:38 freebsd wpa_supplicant[32529]: wlan0: CTRL-EVENT-DISCONNECTED bssid=70:d9:31:0e:2c:00 reason=3 locally_generated=1
Jun  9 23:22:08 freebsd wpa_supplicant[32529]: wlan0: Trying to associate with 70:d9:31:0e:2c:00 (SSID='WHU-STU' freq=2432 MHz)
Jun  9 23:22:08 freebsd wpa_supplicant[32529]: wlan0: Associated with 70:d9:31:0e:2c:00
Jun  9 23:22:08 freebsd wpa_supplicant[32529]: wlan0: CTRL-EVENT-CONNECTED - Connection to 70:d9:31:0e:2c:00 completed [id=1 id_str=]
Jun  9 23:22:09 freebsd dhclient[32836]: send_packet: No buffer space available
Jun  9 23:23:15 freebsd dhclient[32907]: dhclient already running, pid: 32840.
Jun  9 23:23:15 freebsd dhclient[32907]: exiting.
Jun  9 23:23:24 freebsd dhclient[32836]: send_packet: No buffer space available
Jun  9 23:23:27 freebsd dhclient[32922]: New IP Address (wlan0): 10.134.135.174
Jun  9 23:23:27 freebsd dhclient[32926]: New Subnet Mask (wlan0): 255.255.128.0
Jun  9 23:23:27 freebsd dhclient[32930]: New Broadcast Address (wlan0): 10.134.255.255
Jun  9 23:23:27 freebsd dhclient[32934]: New Routers (wlan0): 10.134.255.254
Jun  9 23:23:28 freebsd dhclient[32940]: New Routers (wlan0): 10.134.255.254
Jun  9 23:23:39 freebsd dhclient[33014]: send_packet: No buffer space available
Jun  9 23:24:59 freebsd dhclient[33079]: New IP Address (wlan0): 10.134.135.174
Jun  9 23:24:59 freebsd dhclient[33083]: New Subnet Mask (wlan0): 255.255.128.0
Jun  9 23:24:59 freebsd dhclient[33087]: New Broadcast Address (wlan0): 10.134.255.255
Jun  9 23:24:59 freebsd dhclient[33091]: New Routers (wlan0): 10.134.255.254
Jun  9 23:25:00 freebsd dhclient[33098]: New Routers (wlan0): 10.134.255.254
Comment 50 Cy Schubert freebsd_committer freebsd_triage 2022-06-09 16:22:58 UTC
Understanding that the patch either doesn't fix the problem or it was incorrectly applied, built, or installed, reassigning to network.

Having set up one of my wireless networks as an open hotspot, I cannot reproduce the problem here.
Comment 51 rkoberman 2022-06-11 01:57:49 UTC
The main indicator of the problem is that the mode is set to the rather unlikely value of DS/1Mb. This has only been reported on attempts to connect to public access APs. In one case I believe that it was a campus network. In my case, it has been several hotels while my home router works fine as does the hotspot on my phone (Android).

I was seeing this while running 13.1-stable three weeks ago on a system built from sources of May 2 so it still is probably an issue in STABLE. It was not an issue on 13.0-RELEASE. I was seeing this as far back as early December of last year.

If I get back to such a situation, I will try to bisect. I have no idea when this might be.
Comment 52 Jaskie 2022-06-11 02:13:12 UTC
(In reply to rkoberman from comment #51)

Yes, it is indeed a campus network I am working with.
And I already did a bisect, see comment 44 in this PR for details.
Comment 53 Cy Schubert freebsd_committer freebsd_triage 2022-06-11 23:09:09 UTC
As per my private email, please attach the core file from the special binary I sent you to this PR. Make sure it's compressed using xz -9 to make it as small as possible. The complete instructions are in my email to you.
Comment 54 Cy Schubert freebsd_committer freebsd_triage 2022-06-11 23:12:06 UTC
For anyone following on, the ioctl sent to wlan0 has an op of -1. The SIGABRT core dump will allow me to follow the stack to the routine that sets op to -1, hence the invalid argument.
Comment 55 Cy Schubert freebsd_committer freebsd_triage 2022-06-13 14:12:40 UTC
As per our private email thread, were you able to run the wpa_supplicant provided that will produce a core dump?
Comment 56 Cy Schubert freebsd_committer freebsd_triage 2022-06-13 14:39:37 UTC
The current theory is that wpa_supplicant doesn't work with newer Atheros NICs/driver. The SIGABRT core dump will tell us where in the stack the bad argument to ioctl() is generating the bad argument. This is because it can call bsd_set80211() via three separate instruction paths. Without the dump we are guessing.

As Bugzilla has no such status field as "pending customer" like many commercial bug tracking systems, stating here "pending customer."
Comment 57 Adrian Chadd freebsd_committer freebsd_triage 2022-06-13 15:35:17 UTC
... I don't think the AR9285 stuff has changed in a long time. And most of the ioctls are handed by net80211 first.

Do you need me to jump in and dig into it a bit more? Or are y'all ok?
Comment 58 Jaskie 2022-06-13 15:41:51 UTC
(In reply to Adrian Chadd from comment #57)

By all means, jupm in please. I am currently trying to get a coredump file using the wpa Cy Schubert provided. I have in /etc/sysctl.conf:

kern.corefile=/var/coredumps/%U_%N.core
kern.coredump=1
kern.sugid_coredump=1

ulimit -c reported unlimited But when I ran service netif start or wpa_supplicant -D bsd -i wlan0 -c /etc/wpa_supplicant.conf, no coredump file was found. I checked root home, regular user home, / and /var/coredumps,
/var/crash, no where could I find a coredump file.
Comment 59 J.R. Oldroyd 2022-06-13 18:53:03 UTC
One additional data point.

I am now at a location with a laptop with an AR9280 chip.  I just did a 13.1 upgrade on it.

This one associates just fine with the hostap here, Ethernet MCS mode 11ng.

I realize it's hard to compare apples and oranges (different AR928x chips and different hostaps) but figured I'd post the result.
Comment 60 Cy Schubert freebsd_committer freebsd_triage 2022-06-13 19:26:35 UTC
Considering you can't get a dump, let's see if the ath(4) driver can give us a hint.

Run this DTrace script prior to starting wlan0.

#!/usr/sbin/dtrace -s

fbt::ath_ioctl:entry {
        print(*args[0]);
        stack();
}

fbt::ath_ioctl:return {
        print(arg0);
}
Comment 61 Adrian Chadd freebsd_committer freebsd_triage 2022-06-13 21:21:22 UTC
hey wait a sec, I'm seeing different configurations between wpa-supplicant on 13.0 and 13.1. (ie, out of ifconfig.)

the fact ifconfig isn't saying MANUAL for roam is a big red flag to me. One of the things wpa supplicant does when it's running correctly is literally do /that/.

So did bz/cy change the wpa supplicant API between 13.0 and 13.1? Something tells me they've turned on some options, updated the code, etc and only tested with a couple of NICs rather than all of them, and have broken some stuff.
Comment 62 Bjoern A. Zeeb freebsd_committer freebsd_triage 2022-06-13 21:26:15 UTC
(In reply to Adrian Chadd from comment #61)

Why don't you read a bit backwards; around comment #44 would tell you.
Comment 63 Adrian Chadd freebsd_committer freebsd_triage 2022-06-13 22:30:00 UTC
Ok, it really does look like incomplete testing. :(

Please compile the kernel w/:

IEEE80211_DEBUG
ATH_DEBUG
AH_DEBUG
ATH_DIAGAPI

... I think that should be enough to get all the good debugging that you need.

Then you can use wlandebug to log what's going on when net80211 goes through states:

wlandebug -i wlan0 +assoc +state +scan +mlme

Then run wpa_supplicant and see what happens.

Meanwhile I'll go update my -HEAD laptop again and go setup 13.1-RELEASE somewhere with an ath(4) NIC and see what happens. The only thing that's really ar9285 specific is 1x1 stream 11n + STBC; everything else should be the same on all 2GHz supported ath(4) NICs.
Comment 64 Bjoern A. Zeeb freebsd_committer freebsd_triage 2022-06-13 22:55:45 UTC
(In reply to Adrian Chadd from comment #63)

Again, I think it might be good to remind people that the kernel was ruled out given an 13.0 user land on a 13.1 kernel works just fine if I understand the earlier information correctly.

Also it was reported that copying a 13.0 wpa_supplicant over to 13.1 world makes things work.

It seems based on OOB communication mentioned here, that people are trying to track down a wpa_supplicant/ioctl problem with multiple code paths, so turning on all kernel debugging is probably currently just distracting from the problem.

It's probably also not related to the specific wireless chipset or driver but more likely to the specific BSS as otherwise we'd see a lot more reports and reproducers of that problem?
Comment 65 Adrian Chadd freebsd_committer freebsd_triage 2022-06-13 23:03:54 UTC
Yes, I think it's more likely some ioctl ordering or contents issue. Doing net80211 debugging though may shed some light on what's being done.

(I'd stick a dtrace rule on the net80211 ioctl handle, not the ath ioctl handle, as the ath ioctl handle doesn't do much..)
Comment 66 Cy Schubert freebsd_committer freebsd_triage 2022-06-13 23:21:46 UTC
That's why I asked for a DTrace. The user is unable to capture a dump using the file I gave her that would force wpa_supplicant to dump with the error. There are three different calls to the function that calls ioctl(). I don't know which path it's taking. Hopefully the DTrace would have given me enough of a clue to guess which of the three calls to that function might be the source of the problem. (Apparently DTrace isn't installed on that laptop either... Scratch that idea.)

As to roaming MANUAL, I don't see it. No such problem here nor do I see anything but roaming MANUAL here.
Comment 67 Cy Schubert freebsd_committer freebsd_triage 2022-06-13 23:56:15 UTC
wpa_driver_bsd_scan() calls set80211param() to set IEEE80211_IOC_ROAMING to IEEE80211_ROAMING_MANUAL immediately after setting mediaopt to STA. This is unconditional and if it fails it will issue a message to syslog. All this is passed to bsd_set80211() which calls ioctl at line 120:

        if (ioctl(drv->global->sock, SIOCS80211, &ireq) < 0) {
                wpa_printf(MSG_ERROR, "ioctl[SIOCS80211, op=%u, val=%u, "
                           "arg_len=%u]: %s", op, val, arg_len,
                           strerror(errno));
                return -1;
        }

It is this error message we see in syslog, which may well be that it's failing to set IEEE80211_ROAMING_MANUAL. A backtrace would be able to confirm this.

The interesting thing we notice is that op contains various values. I've seen -1 and 20 so far. op is passed to bsd_set80211() via set80211var() and set80211param(), each of which is called by numerous other callers. strerrorr(errno) is "invalid argument."

The question remains, which caller is responsible for this? Once the user can obtain a dump, the backtrace will tell us which calling path is used and we can discover where opt and val are set, because both of these are set many frames above this frame, thus will be on the stack.

The user will need to provide wpa_supplicant.core from the binary I sent her -- because I put an abort() in place of the return.
Comment 68 Cy Schubert freebsd_committer freebsd_triage 2022-06-14 03:46:16 UTC
The ioctl() error is likely due to bsd_del_key().

Can you please add -dd to wpa_supplicant_flags= in rc.conf. Then grep wpa_supplicant /var/log/messages.
Comment 69 Jaskie 2022-06-14 05:06:27 UTC
Created attachment 234669 [details]
dtrace output
Comment 70 Jaskie 2022-06-14 05:08:28 UTC
(In reply to Jaskie from comment #69)

I am using the wpa Cy Schubert prvided in this test. wpa_supplicant_flags="-dd" was added to rc.conf.

# service netif start
Created wlan(4) interfaces: wlan0.                                                                                                                                                         
Starting wpa_supplicant.                                                                                                                                                                   
wpa_supplicant v2.10                                                                                                                                                                       
Successfully initialized wpa_supplicant                                                                                                                                                    
Initializing interface 'wlan0' conf '/etc/wpa_supplicant.conf' driver 'bsd' ctrl_interface 'N/A' bridge 'N/A'                                                                              
Configuration file '/etc/wpa_supplicant.conf' -> '/etc/wpa_supplicant.conf'                                                                                                                
Reading configuration file '/etc/wpa_supplicant.conf'                                                                                                                                      
Line: 1 - start of a new network block                                                                                                                                                     
ssid - hexdump_ascii(len=8):                                                                                                                                                               
     57 48 55 2d 57 4c 41 4e                           WHU-WLAN                                                                                                                            
key_mgmt: 0x4                                                                                                                                                                              
Line: 11 - start of a new network block                                                                                                                                                    
ssid - hexdump_ascii(len=7):                                                                                                                                                               
     57 48 55 2d 53 54 55                              WHU-STU                                                                                                                             
key_mgmt: 0x4                                                                                                                                                                              
Line: 18 - start of a new network block                                                                                                                                                    
ssid - hexdump_ascii(len=8):                                                                                                                                                               
     57 68 61 74 54 68 65 46                           WhatTheF                                                                                                                            
PSK (ASCII passphrase) - hexdump_ascii(len=8): [REMOVED]                                                                                                                                   
PSK (from passphrase) - hexdump(len=32): [REMOVED]                                                                                                                                         
Priority group 0                                                                                                                                                                           
   id=0 ssid='WHU-WLAN'                                                                                                                                                                    
   id=1 ssid='WHU-STU'                                                                                                                                                                     
   id=2 ssid='WhatTheF'                                                                                                                                                                    
wpa_driver_bsd_capa: drivercaps=0x4f8def41,cryptocaps=0x0000001f                                                                                                                           
Add interface wlan0 to a new radio N/A                                                                                                                                                     
PTKSA: Initializing                                                                                                                                                                        
wlan0: Failed to attach pkt_type filter                                                                                                                                                    
wlan0: Own MAC address: 1c:4b:d6:ca:3e:ac                                                                                                                                                  
bsd_set_key: alg=0 addr=0x0 key_idx=0 set_tx=0 seq_len=0 key_len=0                                                                                                                         
bsd_del_key: key_idx=0                                                                                                                                                                     
bsd_set_key: alg=0 addr=0x0 key_idx=1 set_tx=0 seq_len=0 key_len=0                                                                                                                         
bsd_del_key: key_idx=1                                                                                                                                                                     
bsd_set_key: alg=0 addr=0x0 key_idx=2 set_tx=0 seq_len=0 key_len=0                                                                                                                         
bsd_del_key: key_idx=2                                                                                                                                                                     
bsd_set_key: alg=0 addr=0x0 key_idx=3 set_tx=0 seq_len=0 key_len=0                                                                                                                         
bsd_del_key: key_idx=3                                                                                                                                                                     
bsd_set_key: alg=0 addr=0x0 key_idx=4 set_tx=0 seq_len=0 key_len=0                                                                                                                         
bsd_del_key: key_idx=4                                                                                                                                                                     
ioctl[SIOCS80211, op=20, val=0, arg_len=7]: Invalid argument                                                                                                                               
Abort trap                                                                                                                                                                                 
/etc/rc.d/wpa_supplicant: WARNING: failed to start wpa_supplicant                                                                                                                          
Starting Network: lo0 alc0 wlan0.
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
        inet6 ::1 prefixlen 128
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2
        inet 127.0.0.1 netmask 0xff000000
        groups: lo
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
alc0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=c3198<VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_MCAST,WOL_MAGIC,VLAN_HWTSO,LINKSTATE>
        ether 48:5b:39:8e:3c:b0
        media: Ethernet autoselect (none)
        status: no carrier
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
wlan0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
        ether 1c:4b:d6:ca:3e:ac
        groups: wlan
        ssid "" channel 5 (2432 MHz 11g ht/20)
        regdomain 96 indoor ecm authmode OPEN privacy OFF txpower 20 bmiss 7
        scanvalid 60 protmode CTS ampdulimit 64k ampdudensity 8 shortgi
        -stbctx stbcrx -ldpc -uapsd wme burst bintval 0
        parent interface: ath0
        media: IEEE 802.11 Wireless Ethernet autoselect (autoselect)
        status: no carrier
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>



# less /var/log/messages
Jun 14 12:53:17 freebsd kernel: wlan0: Ethernet address: 1c:4b:d6:ca:3e:ac
Jun 14 12:53:18 freebsd kernel: lo0: link state changed to UP
Jun 14 12:53:18 freebsd root[22584]: /etc/rc.d/wpa_supplicant: WARNING: failed to start wpa_supplicant
Jun 14 12:53:18 freebsd kernel: pid 22579 (wpa_supplicant), jid 0, uid 0: exited on signal 6

# ~adam/dtrace.sh > ~adam/dt_out.txt
dtrace: script '/home/adam/dtrace.sh' matched 2 probes

The xz -9 compressed dt_out.txt is attached as attachment.
Here is also the output from when I am using 13.0 wpa (where network connects successfully)

# service netif start
Created wlan(4) interfaces: wlan0.                                                                                                                                                         
Starting wpa_supplicant.                                                                                                                                                                   
wpa_supplicant v2.9                                                                                                                                                                        
Successfully initialized wpa_supplicant                                                                                                                                                    
Initializing interface 'wlan0' conf '/etc/wpa_supplicant.conf' driver 'bsd' ctrl_interface 'N/A' bridge 'N/A'                                                                              
Configuration file '/etc/wpa_supplicant.conf' -> '/etc/wpa_supplicant.conf'                                                                                                                
Reading configuration file '/etc/wpa_supplicant.conf'                                                                                                                                      
Line: 1 - start of a new network block                                                                                                                                                     
ssid - hexdump_ascii(len=8):                                                                                                                                                               
     57 48 55 2d 57 4c 41 4e                           WHU-WLAN                                                                                                                            
key_mgmt: 0x4                                                                                                                                                                              
Line: 11 - start of a new network block                                                                                                                                                    
ssid - hexdump_ascii(len=7):                                                                                                                                                               
     57 48 55 2d 53 54 55                              WHU-STU                                                                                                                             
key_mgmt: 0x4                                                                                                                                                                              
Line: 18 - start of a new network block                                                                                                                                                    
ssid - hexdump_ascii(len=8):                                                                                                                                                               
     57 68 61 74 54 68 65 46                           WhatTheF                                                                                                                            
PSK (ASCII passphrase) - hexdump_ascii(len=8): [REMOVED]                                                                                                                                   
PSK (from passphrase) - hexdump(len=32): [REMOVED]                                                                                                                                         
Priority group 0                                                                                                                                                                           
   id=0 ssid='WHU-WLAN'                                                                                                                                                                    
   id=1 ssid='WHU-STU'                                                                                                                                                                     
   id=2 ssid='WhatTheF'                                                                                                                                                                    
wpa_driver_bsd_capa: drivercaps=0x4f8def41,cryptocaps=0x0000001f                                                                                                                           
Add interface wlan0 to a new radio N/A                                                                                                                                                     
wlan0: Failed to attach pkt_type filter                                                                                                                                                    
wlan0: Own MAC address: 1c:4b:d6:ca:3e:ac                                                                                                                                                  
bsd_set_key: alg=0 addr=0x0 key_idx=0 set_tx=0 seq_len=0 key_len=0                                                                                                                         
bsd_del_key: key_idx=0                                                                                                                                                                     
bsd_set_key: alg=0 addr=0x0 key_idx=1 set_tx=0 seq_len=0 key_len=0                                                                                                                         
bsd_del_key: key_idx=1                                                                                                                                                                     
bsd_set_key: alg=0 addr=0x0 key_idx=2 set_tx=0 seq_len=0 key_len=0                                                                                                                         
bsd_del_key: key_idx=2                                                                                                                                                                     
bsd_set_key: alg=0 addr=0x0 key_idx=3 set_tx=0 seq_len=0 key_len=0                                                                                                                         
bsd_del_key: key_idx=3                                                                                                                                                                     
bsd_set_key: alg=0 addr=0x0 key_idx=4 set_tx=0 seq_len=0 key_len=0                                                                                                                         
bsd_del_key: key_idx=4                                                                                                                                                                     
ioctl[SIOCS80211, op=20, val=0, arg_len=7]: Invalid argument                                                                                                                               
bsd_set_key: alg=0 addr=0x0 key_idx=5 set_tx=0 seq_len=0 key_len=0                                                                                                                         
bsd_del_key: key_idx=5
ioctl[SIOCS80211, op=20, val=0, arg_len=7]: Invalid argument
wpa_driver_bsd_set_countermeasures: enabled=0
wlan0: RSN: flushing PMKID list in the driver
wlan0: Setting scan request: 0.100000 sec
wlan0: WPS: UUID based on MAC address: 8f102e08-5289-5b2f-a07e-bbde503569e6
ENGINE: Loading builtin engines
ENGINE: Loading builtin engines
EAPOL: SUPP_PAE entering state DISCONNECTED
EAPOL: Supplicant port status: Unauthorized
EAPOL: KEY_RX entering state NO_KEY_RECEIVE
EAPOL: SUPP_BE entering state INITIALIZE
EAP: EAP entering state DISABLED
wlan0: Added interface wlan0
wlan0: State: DISCONNECTED -> DISCONNECTED
Daemonize..
Starting Network: lo0 alc0 wlan0.
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
        inet6 ::1 prefixlen 128
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2
        inet 127.0.0.1 netmask 0xff000000
        groups: lo
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
alc0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=c3198<VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_MCAST,WOL_MAGIC,VLAN_HWTSO,LINKSTATE>
        ether 48:5b:39:8e:3c:b0
        media: Ethernet autoselect (none)
        status: no carrier
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
wlan0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        ether 1c:4b:d6:ca:3e:ac
        inet 10.134.140.105 netmask 0xffff8000 broadcast 10.134.255.255
        groups: wlan
        ssid WHU-STU channel 5 (2432 MHz 11g ht/20) bssid 70:d9:31:0e:2c:00
        regdomain 96 indoor ecm authmode OPEN privacy OFF txpower 20 bmiss 7
        scanvalid 60 protmode CTS ampdulimit 64k shortgi -stbctx stbcrx -ldpc
        -uapsd wme burst roaming MANUAL bintval 300
        parent interface: ath0
        media: IEEE 802.11 Wireless Ethernet MCS mode 11ng
        status: associated
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
Comment 71 Cy Schubert freebsd_committer freebsd_triage 2022-06-14 13:35:24 UTC
#1. I still don't understand why your system will not produce a core dump on abort():

- what does sysctl kern.coredump say?
- what is ullimit -c?
- what is df -h /?

#2. Add priority=100, or give it a higher priority than any other entry in wpa_spplicant.conf.
Comment 72 Jaskie 2022-06-14 14:21:29 UTC
(In reply to Cy Schubert from comment #71)

 sysctl kern.coredump reports 1
 ulimit -c reports unlimited
 df -h / shows there is 200G+ free space

priority=100 was added to WHU-STU block
Comment 73 Cy Schubert freebsd_committer freebsd_triage 2022-06-14 14:38:13 UTC
And...
Comment 74 Jaskie 2022-06-14 14:59:23 UTC
Created attachment 234682 [details]
dtrace out2
Comment 75 Jaskie 2022-06-14 15:01:48 UTC
(In reply to Cy Schubert from comment #73)


Not much difference I noticed:

# service netif start                                                                                                                                       
Starting wpa_supplicant.                                                                                                                                                                   
wpa_supplicant v2.10                                                                                                                                                                       
Successfully initialized wpa_supplicant                                                                                                                                                    
Initializing interface 'wlan0' conf '/etc/wpa_supplicant.conf' driver 'bsd' ctrl_interface 'N/A' bridge 'N/A'                                                                              
Configuration file '/etc/wpa_supplicant.conf' -> '/etc/wpa_supplicant.conf'                                                                                                                
Reading configuration file '/etc/wpa_supplicant.conf'                                                                                                                                      
Line: 1 - start of a new network block                                                                                                                                                     
ssid - hexdump_ascii(len=8):                                                                                                                                                               
     57 48 55 2d 57 4c 41 4e                           WHU-WLAN                                                                                                                            
key_mgmt: 0x4                                                                                                                                                                              
Line: 11 - start of a new network block                                                                                                                                                    
ssid - hexdump_ascii(len=7):                                                                                                                                                               
     57 48 55 2d 53 54 55                              WHU-STU                                                                                                                             
key_mgmt: 0x4                                                                                                                                                                              
priority=100 (0x64)                                                                                                                                                                        
Line: 19 - start of a new network block                                                                                                                                                    
ssid - hexdump_ascii(len=8):                                                                                                                                                               
     57 68 61 74 54 68 65 46                           WhatTheF                                                                                                                            
PSK (ASCII passphrase) - hexdump_ascii(len=8): [REMOVED]                                                                                                                                   
PSK (from passphrase) - hexdump(len=32): [REMOVED]                                                                                                                                         
Priority group 100                                                                                                                                                                         
   id=1 ssid='WHU-STU'                                                                                                                                                                     
Priority group 0                                                                                                                                                                           
   id=0 ssid='WHU-WLAN'                                                                                                                                                                    
   id=2 ssid='WhatTheF'                                                                                                                                                                    
wpa_driver_bsd_capa: drivercaps=0x4f8def41,cryptocaps=0x0000001f                                                                                                                           
Add interface wlan0 to a new radio N/A                                                                                                                                                     
PTKSA: Initializing                                                                                                                                                                        
wlan0: Failed to attach pkt_type filter                                                                                                                                                    
wlan0: Own MAC address: 1c:4b:d6:ca:3e:ac                                                                                                                                                  
bsd_set_key: alg=0 addr=0x0 key_idx=0 set_tx=0 seq_len=0 key_len=0                                                                                                                         
bsd_del_key: key_idx=0                                                                                                                                                                     
bsd_set_key: alg=0 addr=0x0 key_idx=1 set_tx=0 seq_len=0 key_len=0                                                                                                                         
bsd_del_key: key_idx=1                                                                                                                                                                     
bsd_set_key: alg=0 addr=0x0 key_idx=2 set_tx=0 seq_len=0 key_len=0                                                                                                                         
bsd_del_key: key_idx=2                                                                                                                                                                     
bsd_set_key: alg=0 addr=0x0 key_idx=3 set_tx=0 seq_len=0 key_len=0                                                                                                                         
bsd_del_key: key_idx=3                                                                                                                                                                     
bsd_set_key: alg=0 addr=0x0 key_idx=4 set_tx=0 seq_len=0 key_len=0                                                                                                                         
bsd_del_key: key_idx=4
ioctl[SIOCS80211, op=20, val=0, arg_len=7]: Invalid argument                                                                                                                               
Abort trap
/etc/rc.d/wpa_supplicant: WARNING: failed to start wpa_supplicant
Starting Network: lo0 alc0 wlan0.
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
        inet6 ::1 prefixlen 128
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2
        inet 127.0.0.1 netmask 0xff000000
        groups: lo
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
alc0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=c3198<VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_MCAST,WOL_MAGIC,VLAN_HWTSO,LINKSTATE>
        ether 48:5b:39:8e:3c:b0
        media: Ethernet autoselect (none)
        status: no carrier
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
wlan0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
        ether 1c:4b:d6:ca:3e:ac
        groups: wlan
        ssid "" channel 5 (2432 MHz 11g ht/20) 
        regdomain 96 indoor ecm authmode OPEN privacy OFF txpower 20 bmiss 7
        scanvalid 60 protmode CTS ampdulimit 64k ampdudensity 8 shortgi
        -stbctx stbcrx -ldpc -uapsd wme burst bintval 0
        parent interface: ath0
        media: IEEE 802.11 Wireless Ethernet autoselect (autoselect)
        status: no carrier
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
root@freebsd: /usr/sbin @22:52 # dhclient wlan0
wlan0: no link .............. giving up
# ifconfig wlan0
wlan0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
        ether 1c:4b:d6:ca:3e:ac
        groups: wlan
        ssid "" channel 5 (2432 MHz 11g ht/20)
        regdomain 96 indoor ecm authmode OPEN privacy OFF txpower 20 bmiss 7
        scanvalid 60 protmode CTS ampdulimit 64k ampdudensity 8 shortgi
        -stbctx stbcrx -ldpc -uapsd wme burst bintval 0
        parent interface: ath0
        media: IEEE 802.11 Wireless Ethernet autoselect (autoselect)
        status: no carrier
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>


# less /var/log/messages
Jun 14 22:52:19 freebsd root[25861]: /etc/rc.d/wpa_supplicant: WARNING: failed to start wpa_supplicant
Jun 14 22:52:19 freebsd kernel: pid 25857 (wpa_supplicant), jid 0, uid 0: exited on signal 6

And still didn't see any coredump files
Comment 76 Cy Schubert freebsd_committer freebsd_triage 2022-06-14 15:28:11 UTC
DTrace output doesn't help anymore because DTrace won't dereference a pointer to void *.
Comment 77 Jaskie 2022-06-14 15:40:41 UTC
(In reply to Cy Schubert from comment #76)

But I didn't get any coredump files

# sysctl kern.corefile
kern.corefile: /var/coredumps/%U/%N.core
# sysctl kern.coredump
kern.coredump: 1

However, /var/coredumps is always emtpy.
Comment 78 Bjoern A. Zeeb freebsd_committer freebsd_triage 2022-06-14 15:46:31 UTC
(In reply to Jaskie from comment #77)

Try changing:

kern.corefile: /var/coredumps/%U/%N.core

to

sysctl kern.corefile="/var/coredumps/%U-%N.core"

to avoid problems with another subdir maybe?
Comment 79 Cy Schubert freebsd_committer freebsd_triage 2022-06-14 15:47:15 UTC
ls -lh /var/coredumps, please.
Comment 80 Cy Schubert freebsd_committer freebsd_triage 2022-06-14 15:53:37 UTC
(In reply to Bjoern A. Zeeb from comment #78)

My point exactly. /var/coredumps/0 probably does not exist and the kernel will not create the directory. It will simply bail. I think she's trying to duplicate the functionality of Linux's abrtd using this sysctl. (We should probably write something like that, to perform some basic triage like rc.d/savecore does but for userland core dumps. That's for another day and another project.)
Comment 81 Jaskie 2022-06-15 00:40:41 UTC
Created attachment 234695 [details]
coredump from wpa_supplicant
Comment 82 Jaskie 2022-06-15 00:41:58 UTC
(In reply to Bjoern A. Zeeb from comment #78)

yes yes yes exactly. I changed this and get the coredump file. I've uploaded it in attachment. Thanks! hope this helps.
Comment 83 Cy Schubert freebsd_committer freebsd_triage 2022-06-17 04:41:25 UTC
This could be a timing issue. Can you please add the following to rc.conf,

synchronous_dhclient="YES"
Comment 84 Jaskie 2022-06-17 05:52:16 UTC
(In reply to Cy Schubert from comment #83)

I added synchronous_dhclient="YES" to /etc/rc/conf, copied wpa_supplicant v2.10 in place, network was not working. dt_out2.txt.xz is uploaded as attachment.

# service netif start
Created wlan(4) interfaces: wlan0.                                                                                                                                                         
Starting dhclient.                                                                                                                                                                         
alc0: no link .............. giving up                                                                                                                                                     
/etc/rc.d/dhclient: WARNING: failed to start dhclient                                                                                                                                      
Starting wpa_supplicant.                                                                                                                                                                   
wpa_supplicant v2.10                                                                                                                                                                       
Successfully initialized wpa_supplicant                                                                                                                                                    
Initializing interface 'wlan0' conf '/etc/wpa_supplicant.conf' driver 'bsd' ctrl_interface 'N/A' bridge 'N/A'                                                                              
Configuration file '/etc/wpa_supplicant.conf' -> '/etc/wpa_supplicant.conf'                                                                                                                
Reading configuration file '/etc/wpa_supplicant.conf'                                                                                                                                      
Line: 1 - start of a new network block                                                                                                                                                     
ssid - hexdump_ascii(len=8):                                                                                                                                                               
     57 48 55 2d 57 4c 41 4e                           WHU-WLAN                                                                                                                            
key_mgmt: 0x4                                                                                                                                                                              
Line: 11 - start of a new network block                                                                                                                                                    
ssid - hexdump_ascii(len=7):                                                                                                                                                               
     57 48 55 2d 53 54 55                              WHU-STU                                                                                                                             
key_mgmt: 0x4                                                                                                                                                                              
priority=100 (0x64)                                                                                                                                                                        
Line: 19 - start of a new network block                                                                                                                                                    
ssid - hexdump_ascii(len=8):
     57 68 61 74 54 68 65 46                           WhatTheF        
PSK (ASCII passphrase) - hexdump_ascii(len=8): [REMOVED]

# ifconfig wlan0
wlan0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        ether 1c:4b:d6:ca:3e:ac
        inet 0.0.0.0 netmask 0xff000000 broadcast 255.255.255.255
        groups: wlan
        ssid WHU-STU channel 5 (2432 MHz 11g ht/20) bssid 70:d9:31:0e:2c:00
        regdomain 96 indoor ecm authmode WPA privacy MIXED deftxkey UNDEF
        txpower 20 bmiss 7 scanvalid 60 protmode CTS ampdulimit 64k shortgi
        -stbctx stbcrx -ldpc -uapsd wme burst roaming MANUAL bintval 300
        parent interface: ath0
        media: IEEE 802.11 Wireless Ethernet DS/1Mbps mode 11ng
        status: associated
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>

PSK (from passphrase) - hexdump(len=32): [REMOVED]
Priority group 100
   id=1 ssid='WHU-STU'
Priority group 0
   id=0 ssid='WHU-WLAN'
   id=2 ssid='WhatTheF'
wpa_driver_bsd_capa: drivercaps=0x4f8def41,cryptocaps=0x0000001f
Add interface wlan0 to a new radio N/A
PTKSA: Initializing
wlan0: Failed to attach pkt_type filter
wlan0: Own MAC address: 1c:4b:d6:ca:3e:ac
bsd_set_key: alg=0 addr=0x0 key_idx=0 set_tx=0 seq_len=0 key_len=0
bsd_del_key: key_idx=0
bsd_set_key: alg=0 addr=0x0 key_idx=1 set_tx=0 seq_len=0 key_len=0
bsd_del_key: key_idx=1
bsd_set_key: alg=0 addr=0x0 key_idx=2 set_tx=0 seq_len=0 key_len=0
bsd_del_key: key_idx=2
bsd_set_key: alg=0 addr=0x0 key_idx=3 set_tx=0 seq_len=0 key_len=0
bsd_del_key: key_idx=3
bsd_set_key: alg=0 addr=0x0 key_idx=4 set_tx=0 seq_len=0 key_len=0
bsd_del_key: key_idx=4                                                                                                                                                                     
ioctl[SIOCS80211, op=20, val=0, arg_len=7]: Invalid argument                                                                                                                               
bsd_set_key: alg=0 addr=0x0 key_idx=5 set_tx=0 seq_len=0 key_len=0                                                                                                                         
bsd_del_key: key_idx=5                                                                                                                                                                     
ioctl[SIOCS80211, op=20, val=0, arg_len=7]: Invalid argument
wpa_driver_bsd_set_countermeasures: enabled=0
wlan0: RSN: flushing PMKID list in the driver
wlan0: Setting scan request: 0.100000 sec
TDLS: TDLS operation not supported by driver
TDLS: Driver uses internal link setup
TDLS: Driver does not support TDLS channel switching
wlan0: WPS: UUID based on MAC address: 8f102e08-5289-5b2f-a07e-bbde503569e6
ENGINE: Loading builtin engines
ENGINE: Loading builtin engines
EAPOL: SUPP_PAE entering state DISCONNECTED
EAPOL: Supplicant port status: Unauthorized
EAPOL: KEY_RX entering state NO_KEY_RECEIVE
EAPOL: SUPP_BE entering state INITIALIZE
EAP: EAP entering state DISABLED
MBO: Update non-preferred channels, non_pref_chan=N/A
wlan0: Added interface wlan0
wlan0: State: DISCONNECTED -> DISCONNECTED
Daemonize..
Starting dhclient.
wlan0: no link .... got link
Starting Network: lo0 alc0 wlan0.
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
        inet6 ::1 prefixlen 128
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2
        inet 127.0.0.1 netmask 0xff000000
        groups: lo
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
alc0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=c3198<VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_MCAST,WOL_MAGIC,VLAN_HWTSO,LINKSTATE>
        ether 48:5b:39:8e:3c:b0
        media: Ethernet autoselect (none)
        status: no carrier
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
wlan0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        ether 1c:4b:d6:ca:3e:ac
        inet 0.0.0.0 netmask 0xff000000 broadcast 255.255.255.255
        groups: wlan
        ssid WHU-STU channel 5 (2432 MHz 11g ht/20) bssid 70:d9:31:0e:2c:00
        regdomain 96 indoor ecm authmode WPA privacy MIXED deftxkey UNDEF
        txpower 20 bmiss 7 scanvalid 60 protmode CTS ampdulimit 64k shortgi
        -stbctx stbcrx -ldpc -uapsd wme burst roaming MANUAL bintval 300
        parent interface: ath0
        media: IEEE 802.11 Wireless Ethernet DS/1Mbps mode 11ng
        status: associated
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>


# less /var/log/messages
Jun 17 13:34:07 freebsd kernel: wlan0: Ethernet address: 1c:4b:d6:ca:3e:ac
Jun 17 13:34:07 freebsd kernel: lo0: link state changed to UP
Jun 17 13:34:19 freebsd adam[2850]: /etc/rc.d/dhclient: WARNING: failed to start dhclient
Jun 17 13:34:23 freebsd kernel: wlan0: link state changed to UP
Jun 17 13:34:23 freebsd dhclient[2919]: send_packet: No buffer space available
Jun 17 13:35:03 freebsd syslogd: last message repeated 5 times
Jun 17 13:35:26 freebsd syslogd: last message repeated 2 times
Jun 17 13:35:39 freebsd dhclient[3003]: New IP Address (wlan0): 10.134.140.105
Jun 17 13:35:39 freebsd dhclient[3007]: New Subnet Mask (wlan0): 255.255.128.0
Jun 17 13:35:39 freebsd dhclient[3011]: New Broadcast Address (wlan0): 10.134.255.255
Jun 17 13:35:39 freebsd dhclient[3015]: New Routers (wlan0): 10.134.255.254
Jun 17 13:35:40 freebsd dhclient[3020]: New Routers (wlan0): 10.134.255.254

wpa_supplicant v2.9 from 13.0 worked as expected.
Comment 85 Jaskie 2022-06-17 05:53:29 UTC
Created attachment 234745 [details]
dtrace from wpa v2.10

dtrace from wpa v2.10
synchronous_dhclient="YES"
wpa_supplicant_flags="-dd"
Comment 86 Cy Schubert freebsd_committer freebsd_triage 2022-06-17 12:53:05 UTC
We don't need anymore dtrace listings.

I assume it also worked with wpa_supplicant 2.10 on 13.1.
Comment 87 Cy Schubert freebsd_committer freebsd_triage 2022-06-17 12:53:50 UTC
Setting synchronous dhclient fixed the problem.
Comment 88 Adrian Chadd freebsd_committer freebsd_triage 2022-06-17 16:04:05 UTC
wait a sec, before you close it, was the actual crash in wpa_supplicant resolved?
Comment 89 Cy Schubert freebsd_committer freebsd_triage 2022-06-17 16:14:57 UTC
Apparently so.
Comment 90 Cy Schubert freebsd_committer freebsd_triage 2022-06-17 16:49:30 UTC
can you provide full -dd output using the 13.1 wpa_supplicant. DTrace output is useless ATM.
Comment 91 Jaskie 2022-06-18 01:02:01 UTC
(In reply to Cy Schubert from comment #90)

No, it was NOT WORKING. I would've mentioned if it was fixed after adding synchronous_dhclient="YES".
The only thing changed as I noticed was that now wlan0 claimed to be associated, despite that there was no IP address as ifconfig reported.

This is from wpa 2.10 but I am not sure it's the vanilla 13.1 one or the one I compiled from source:

# sha512sum wpa_supplicant
f2112c9a445c27bbc7cc73064a2c68240d8cd32747a7803941a62313103a869418ee24c68aa1ffd7659c3e3c425decd895d123424a50ab0e9b2d73545b69e3f5

Output from service netif start, wpa with -dd, and ifconfig:

# service netif start                                                                                                                                        
Created wlan(4) interfaces: wlan0.                                                                                                                                                         
Starting dhclient.                                                                                                                                                                         
alc0: no link .............. giving up                                                                                                                                                     
/etc/rc.d/dhclient: WARNING: failed to start dhclient                                                                                                                                      
Starting wpa_supplicant.                                                                                                                                                                   
wpa_supplicant v2.10                                                                                                                                                                       
Successfully initialized wpa_supplicant                                                                                                                                                    
Initializing interface 'wlan0' conf '/etc/wpa_supplicant.conf' driver 'bsd' ctrl_interface 'N/A' bridge 'N/A'                                                                              
Configuration file '/etc/wpa_supplicant.conf' -> '/etc/wpa_supplicant.conf'                                                                                                                
Reading configuration file '/etc/wpa_supplicant.conf'                                                                                                                                      
Line: 1 - start of a new network block                                                                                                                                                     
ssid - hexdump_ascii(len=8):                                                                                                                                                               
     57 48 55 2d 57 4c 41 4e                           WHU-WLAN                                                                                                                            
key_mgmt: 0x4                                                                                                                                                                              
Line: 11 - start of a new network block                                                                                                                                                    
ssid - hexdump_ascii(len=7):                                                                                                                                                               
     57 48 55 2d 53 54 55                              WHU-STU                                                                                                                             
key_mgmt: 0x4                                                                                                                                                                              
priority=100 (0x64)                                                                                                                                                                        
Line: 19 - start of a new network block                                                                                                                                                    
ssid - hexdump_ascii(len=8):                                                                                                                                                               
     57 68 61 74 54 68 65 46                           WhatTheF                                                                                                                            
PSK (ASCII passphrase) - hexdump_ascii(len=8): [REMOVED]                                                                                                                                   
PSK (from passphrase) - hexdump(len=32): [REMOVED]                                                                                                                                         
Priority group 100                            
   id=1 ssid='WHU-STU'                        
Priority group 0                              
   id=0 ssid='WHU-WLAN'                       
   id=2 ssid='WhatTheF'                       
wpa_driver_bsd_capa: drivercaps=0x4f8def41,cryptocaps=0x0000001f                                                                                                                           
Add interface wlan0 to a new radio N/A                                                       
PTKSA: Initializing                           
wlan0: Failed to attach pkt_type filter                                                      
wlan0: Own MAC address: 1c:4b:d6:ca:3e:ac                                                    
bsd_set_key: alg=0 addr=0x0 key_idx=0 set_tx=0 seq_len=0 key_len=0                                                                                                                         
bsd_del_key: key_idx=0                        
bsd_set_key: alg=0 addr=0x0 key_idx=1 set_tx=0 seq_len=0 key_len=0                                                                                                                         
bsd_del_key: key_idx=1                        
bsd_set_key: alg=0 addr=0x0 key_idx=2 set_tx=0 seq_len=0 key_len=0                                                                                                                         
bsd_del_key: key_idx=2                        
bsd_set_key: alg=0 addr=0x0 key_idx=3 set_tx=0 seq_len=0 key_len=0                                                                                                                         
bsd_del_key: key_idx=3                        
bsd_set_key: alg=0 addr=0x0 key_idx=4 set_tx=0 seq_len=0 key_len=0
bsd_del_key: key_idx=4                        
ioctl[SIOCS80211, op=20, val=0, arg_len=7]: Invalid argument                                 
bsd_set_key: alg=0 addr=0x0 key_idx=5 set_tx=0 seq_len=0 key_len=0                                                                                                                         
bsd_del_key: key_idx=5                        
ioctl[SIOCS80211, op=20, val=0, arg_len=7]: Invalid argument                                 
wpa_driver_bsd_set_countermeasures: enabled=0                                                
wlan0: RSN: flushing PMKID list in the driver                                                
wlan0: Setting scan request: 0.100000 sec                                                    
TDLS: TDLS operation not supported by driver                                                 
TDLS: Driver uses internal link setup                                                        
TDLS: Driver does not support TDLS channel switching                                         
wlan0: WPS: UUID based on MAC address: 8f102e08-5289-5b2f-a07e-bbde503569e6                                                                                                                
ENGINE: Loading builtin engines                                                              
ENGINE: Loading builtin engines                                                              
EAPOL: SUPP_PAE entering state DISCONNECTED                                                  
EAPOL: Supplicant port status: Unauthorized                                                  
EAPOL: KEY_RX entering state NO_KEY_RECEIVE                                                  
EAPOL: SUPP_BE entering state INITIALIZE                                                     
EAP: EAP entering state DISABLED                                                             
MBO: Update non-preferred channels, non_pref_chan=N/A                                        
wlan0: Added interface wlan0                  
wlan0: State: DISCONNECTED -> DISCONNECTED                                                   
Daemonize..                                   
Starting dhclient.                            
wlan0: no link .... got link                  
Starting Network: lo0 alc0 wlan0.                                                            
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384                                                                                                                          
        options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>                                                                                                                    
        inet6 ::1 prefixlen 128                                                              
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2                                           
        inet 127.0.0.1 netmask 0xff000000                                                    
        groups: lo                            
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>                                            
alc0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500                                                                                                                 
        options=c3198<VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_MCAST,WOL_MAGIC,VLAN_HWTSO,LINKSTATE>                                                                                   
        ether 48:5b:39:8e:3c:b0                                                              
        media: Ethernet autoselect (none)                                                    
        status: no carrier                    
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
wlan0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500                                                                                                                
        ether 1c:4b:d6:ca:3e:ac                                                              
        inet 0.0.0.0 netmask 0xff000000 broadcast 255.255.255.255                                                                                                                          
        groups: wlan                          
        ssid WHU-STU channel 5 (2432 MHz 11g ht/20) bssid 70:d9:31:0e:2c:00                                                                                                                
        regdomain 96 indoor ecm authmode WPA privacy MIXED deftxkey UNDEF                                                                                                                  
        txpower 20 bmiss 7 scanvalid 60 protmode CTS ampdulimit 64k shortgi                                                                                                                
        -stbctx stbcrx -ldpc -uapsd wme burst roaming MANUAL bintval 300                                                                                                                   
        parent interface: ath0                
        media: IEEE 802.11 Wireless Ethernet DS/1Mbps mode 11ng                                                                                                                            
        status: associated                    
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>


# /usr/sbin/wpa_supplicant -dd -B -i wlan0 -c /etc/wpa_supplicant.conf
wpa_supplicant v2.10
Successfully initialized wpa_supplicant
Initializing interface 'wlan0' conf '/etc/wpa_supplicant.conf' driver 'default' ctrl_interface 'N/A' bridge 'N/A'
Configuration file '/etc/wpa_supplicant.conf' -> '/etc/wpa_supplicant.conf'
Reading configuration file '/etc/wpa_supplicant.conf'
Line: 1 - start of a new network block
ssid - hexdump_ascii(len=8):
     57 48 55 2d 57 4c 41 4e                           WHU-WLAN        
key_mgmt: 0x4
Line: 11 - start of a new network block
ssid - hexdump_ascii(len=7):
     57 48 55 2d 53 54 55                              WHU-STU         
key_mgmt: 0x4
priority=100 (0x64)
Line: 19 - start of a new network block
ssid - hexdump_ascii(len=8):
     57 68 61 74 54 68 65 46                           WhatTheF        
PSK (ASCII passphrase) - hexdump_ascii(len=8): [REMOVED]
PSK (from passphrase) - hexdump(len=32): [REMOVED]
Priority group 100
   id=1 ssid='WHU-STU'
Priority group 0
   id=0 ssid='WHU-WLAN'
   id=2 ssid='WhatTheF'
wpa_driver_bsd_capa: drivercaps=0x4f8def41,cryptocaps=0x0000001f
Add interface wlan0 to a new radio N/A
PTKSA: Initializing
wlan0: Failed to attach pkt_type filter
wlan0: Own MAC address: 1c:4b:d6:ca:3e:ac
bsd_set_key: alg=0 addr=0x0 key_idx=0 set_tx=0 seq_len=0 key_len=0
bsd_del_key: key_idx=0
bsd_set_key: alg=0 addr=0x0 key_idx=1 set_tx=0 seq_len=0 key_len=0
bsd_del_key: key_idx=1
bsd_set_key: alg=0 addr=0x0 key_idx=2 set_tx=0 seq_len=0 key_len=0
bsd_del_key: key_idx=2
bsd_set_key: alg=0 addr=0x0 key_idx=3 set_tx=0 seq_len=0 key_len=0
bsd_del_key: key_idx=3
bsd_set_key: alg=0 addr=0x0 key_idx=4 set_tx=0 seq_len=0 key_len=0
bsd_del_key: key_idx=4
ioctl[SIOCS80211, op=20, val=0, arg_len=7]: Invalid argument
bsd_set_key: alg=0 addr=0x0 key_idx=5 set_tx=0 seq_len=0 key_len=0
bsd_del_key: key_idx=5
ioctl[SIOCS80211, op=20, val=0, arg_len=7]: Invalid argument
wpa_driver_bsd_set_countermeasures: enabled=0
wlan0: RSN: flushing PMKID list in the driver
wlan0: Setting scan request: 0.100000 sec
TDLS: TDLS operation not supported by driver
TDLS: Driver uses internal link setup
TDLS: Driver does not support TDLS channel switching
wlan0: WPS: UUID based on MAC address: 8f102e08-5289-5b2f-a07e-bbde503569e6
ENGINE: Loading builtin engines
ENGINE: Loading builtin engines
EAPOL: SUPP_PAE entering state DISCONNECTED
EAPOL: Supplicant port status: Unauthorized
EAPOL: KEY_RX entering state NO_KEY_RECEIVE
EAPOL: SUPP_BE entering state INITIALIZE
EAP: EAP entering state DISABLED
MBO: Update non-preferred channels, non_pref_chan=N/A
wlan0: Added interface wlan0
wlan0: State: DISCONNECTED -> DISCONNECTED
Daemonize..


# ifconfig wlan0                                                
wlan0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        ether 1c:4b:d6:ca:3e:ac
        inet 0.0.0.0 netmask 0xff000000 broadcast 255.255.255.255
        groups: wlan
        ssid WHU-STU channel 5 (2432 MHz 11g ht/20) bssid 70:d9:31:0e:2c:00
        regdomain 96 indoor ecm authmode WPA privacy MIXED deftxkey UNDEF
        txpower 20 bmiss 7 scanvalid 60 protmode CTS ampdulimit 64k shortgi
        -stbctx stbcrx -ldpc -uapsd wme burst roaming MANUAL bintval 300
        parent interface: ath0
        media: IEEE 802.11 Wireless Ethernet DS/1Mbps mode 11ng
        status: associated
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
Comment 92 Cy Schubert freebsd_committer freebsd_triage 2022-06-18 14:18:24 UTC
After it fails, please run,

# killall dhclient
# dhclient wlan0

Also, does your rc.conf contain, ifconfig_wlan0="WPA DHCP" ?
Comment 93 Jaskie 2022-06-18 14:40:07 UTC
(In reply to Cy Schubert from comment #92)

Yes, I have the line in /etc/rc.conf:

# grep wlan0 /etc/rc.conf
wlans_ath0="wlan0"
ifconfig_wlan0="WPA SYNCDHCP"
#ifconfig_wlan0="WPA DHCP"

I tried both SYNCDHCP and DHCP, neither worked.

After service netif start I check with ps -aux and made sure wpa and dhclient was running automatically, but still no IP was assigned to wlan0 as shown by ifconfig though it reported wlan0 as associated. If I kill dhclient and ran it manually, I got the same output as before:

# killall dhclient
# dhclient wlan0
DHCPREQUEST on wlan0 to 255.255.255.255 port 67
DHCPREQUEST on wlan0 to 255.255.255.255 port 67
DHCPREQUEST on wlan0 to 255.255.255.255 port 67
DHCPDISCOVER on wlan0 to 255.255.255.255 port 67 interval 6
DHCPDISCOVER on wlan0 to 255.255.255.255 port 67 interval 7
DHCPDISCOVER on wlan0 to 255.255.255.255 port 67 interval 13
DHCPDISCOVER on wlan0 to 255.255.255.255 port 67 interval 19
DHCPDISCOVER on wlan0 to 255.255.255.255 port 67 interval 12
DHCPDISCOVER on wlan0 to 255.255.255.255 port 67 interval 4
No DHCPOFFERS received.
Trying recorded lease 10.134.140.105
No working leases in persistent database - sleeping.

On the other hand, network is up and fine with wpa from 13.0 with either DHCP or SYNCDHCP as soon as I run service netif start, no need to kill and run dhclient by myself.
Comment 94 Cy Schubert freebsd_committer freebsd_triage 2022-06-18 15:29:19 UTC
What other entries are in your wpa_supplicant.conf?

Appears it's associated with authmode OPEN privacy OFF under wpa_supplicant 2.9 while authmode WPA privacy MIXED deftxkey UNDEF under wpa_supplicant 2.10. This suggests that the AP and the STA have negotiated WPA even though it is not configured as such in wpa_supplicant.conf

I've posted a question on the hostap mailing list.
Comment 95 Jaskie 2022-06-18 15:40:24 UTC
(In reply to Cy Schubert from comment #94)

I posted the content of /etc/wpa_supplicant.conf earlier and didn't made much change since that:

network={
        ssid="WHU-WLAN"
        key_mgmt=NONE
}                                                                                                                                                      

#network={ 
#        ssid="TP-LINK_B1FD"
#       psk="I123c456U789"
#}                                                                                                                                                     #

network={
        ssid="WHU-STU"
        #bssid=fc:b6:98:f6:12:80
        #bssid=70:d9:31:0e:2c:00
        key_mgmt=NONE
        priority=100
}

network={
        ssid="WhatTheF"
        psk="jiangjun"
}

Also, here is content of /etc/rc.conf and /etc/sysctl.conf in case you'll need it.

# cat /etc/rc.conf
zfs_enable="YES"
rc_startmsgs="NO"
hostname="freebsd.asus"

wlans_ath0="wlan0"
ifconfig_wlan0="WPA SYNCDHCP"
#ifconfig_wlan0="WPA DHCP"
background_dhclient="YES"
ifconfig_alc0="DHCP"
synchronous_dhclient="YES"
wpa_supplicant_flags="-dd"

clear_tmp_enable="YES"
clear_tmp_X="YES"
dbus_enable="YES"
dumpdev="AUTO"
syslogd_flags="-ss"

sshd_enable="YES"

sendmail_enable="NO"
sendmail_submit_enable="NO"
sendmail_outbound_enable="NO"
sendmail_msp_queue_enable="NO"

lightdm_enable="NO"
xdm_enable="YES"
moused_enable="YES"

kld_list="nvidia fusefs acpi_asus acpi_asus_wmi acpi_video"


# cat /etc/sysctl.conf 
# $FreeBSD$
#
#  This file is read when going to multi-user and its contents piped thru
#  ``sysctl'' to adjust kernel values.  ``man 5 sysctl.conf'' for details.
#

# Uncomment this to prevent users from seeing information about processes that
# are being run under another UID.
#security.bsd.see_other_uids=0

hw.acpi.lid_switch_state="S3"

kern.corefile=/var/coredumps/%U_%N.core
kern.coredump=1
#kern.sugid_coredump=1
Comment 96 Adrian Chadd freebsd_committer freebsd_triage 2022-06-19 14:29:05 UTC
hm, if this affects multiple people, should we consider rolling back wpa_s until we can figure out what's up?
Comment 97 Cy Schubert freebsd_committer freebsd_triage 2022-06-19 15:02:12 UTC
(In reply to Adrian Chadd from comment #96)

I'm preparing a wpa_supplicant29 port.

Reverting would cause too much churn. Though I may consider reverting only 12 and 13. Reverting would also remove support for WiFi 6 back to WiFi 5.

Once reverted there is no way to MFC back again. Git doesn't support that because the MFC has already been registered in the tree. The patches would need to be reapplied by hand (which could cause other issues and errors) or we never go forward again. I think a wpa29 port would be the best workaround for now.

This AR9285 problem was not discovered for six months on -CURRENT and another four months on -STABLE. It was only discovered in -RELEASE.

This only affects users of AR9285. I understand you have AR9285. Would you be willing to send me a card? Either for laptop or PCI.
Comment 98 commit-hook freebsd_committer freebsd_triage 2022-06-19 16:21:38 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/ports/commit/?id=7150a0c9b1014e445a8266c9080d0bf4738dcc9c

commit 7150a0c9b1014e445a8266c9080d0bf4738dcc9c
Author:     Cy Schubert <cy@FreeBSD.org>
AuthorDate: 2022-06-19 16:15:44 +0000
Commit:     Cy Schubert <cy@FreeBSD.org>
CommitDate: 2022-06-19 16:20:13 +0000

    */*: Bring back wpa_supplicant29 and hostapd29 as new ports

    The current wpa_supplicant and hostapd have an issue with AR9285.
    For the time being bring back wpa_supplicant 2.9 as
    security/wpa_supplicant29 and hostpd 2.9 as net/hostapd29 for those
    cases that have an issue with wpa_supplicant/hostpad2.10 (in base and
    in ports)

    PR:             264238
    MFH:            2022Q2

 net/Makefile                                       |   1 +
 net/hostapd29/Makefile (new)                       |  46 +++
 net/hostapd29/distinfo (new)                       |   9 +
 net/hostapd29/files/config (new)                   | 316 ++++++++++++++++++
 net/hostapd29/files/hostapd.in (new)               |  39 +++
 .../patch-src-l2_packet-l2_packet_freebsd.c (new)  |  14 +
 net/hostapd29/files/patch-src_common_dhcp.h (new)  |  25 ++
 .../files/patch-src_drivers_driver__bsd.c (new)    |  60 ++++
 net/hostapd29/files/patch-src_utils_os.h (new)     |  17 +
 .../files/patch-src_utils_os__unix.c (new)         |  18 +
 .../files/patch-src_wps_wps__upnp.c (new)          |  20 ++
 net/hostapd29/pkg-descr (new)                      |  12 +
 net/hostapd29/pkg-message (new)                    |  10 +
 security/Makefile                                  |   1 +
 security/wpa_supplicant29/Makefile (new)           | 229 +++++++++++++
 security/wpa_supplicant29/distinfo (new)           |  11 +
 security/wpa_supplicant29/files/Packet32.c (new)   | 366 +++++++++++++++++++++
 security/wpa_supplicant29/files/Packet32.h (new)   |  65 ++++
 security/wpa_supplicant29/files/ntddndis.h (new)   |  32 ++
 .../files/patch-src_common_dhcp.h (new)            |  25 ++
 .../files/patch-src_drivers_driver__bsd.c (new)    |  48 +++
 .../files/patch-src_drivers_driver__ndis.c (new)   |  89 +++++
 ...atch-src_l2__packet_l2__packet__freebsd.c (new) |  12 +
 .../files/patch-src_radius_radius__client.c (new)  |  12 +
 .../files/patch-src_wps_wps__upnp.c (new)          |  34 ++
 .../files/patch-wpa__supplicant_Makefile (new)     |  17 +
 .../files/patch-wpa__supplicant_main.c (new)       |  33 ++
 .../patch-wpa__supplicant_wpa__supplicant.c (new)  |  16 +
 .../wpa_supplicant29/files/pkg-message.in (new)    |  11 +
 .../wpa_supplicant29/files/wpa_supplicant.in (new) |  54 +++
 security/wpa_supplicant29/pkg-descr (new)          |  14 +
 security/wpa_supplicant29/pkg-plist (new)          |   5 +
 32 files changed, 1661 insertions(+)
Comment 99 rkoberman 2022-06-19 16:45:12 UTC
Once again, while the title refers to the AR9285, I am seeing the same symptoms on my AX200. Unfortunately, since it seems to only affect connection to OPEN access points, I only see this on the infrequent occasions when I am away from home, usually at a hotel.

I see the same behavior with the AX200 appearing to associate with the AP, but it fails to et an address. Working with the surprisingly cluefull support for whoever Marriott contracts with (he runs Mint Linux at home), he monitored my connection to confirm that they were seeing no packets from my MAC address. On my end, tcpdump saw the DHCP DISCOVER packets going out.

Key indicator is that the connection media seen on the interface is wrong. While I did not always see the same media, it was usually DS/1MB which is clearly wrong. When I connect to my home network, I see a more reasonable OFDM/54Mbps.

Once I have installed the 2.9 port, I will take my laptop down to a Starbucks and/or McDonald's and see what happens there. I can try both 2.9 and 3.0 as well as test with both the AX200 and the Realtek USB interface I was using before iwlwifi was available.
Comment 100 Adrian Chadd freebsd_committer freebsd_triage 2022-06-19 17:20:50 UTC
when you're down there, can you do an "ifconfig -v wlan0 list scan", or "ifconfig wlan0 -v list scan" (I forget which), and "ifconfig -v wlan0" to get a full dump of the scan data and the NIC state? That way I can try to reproduce it with an open AP here.

I'm specifically interested in the protocol handling. I have a feeling that enabling 11n/11ac/11ax in wpa_supplicant may be clashing with some stuff done in net80211, but the only real way to see what's going on there is to try and set it up here to reproduce it.

And cy@, yeah, I can throw you a mini-pcie AR9285. I have squillions of them here. :) They're a good test case for 1x1 + STBC (and AR9485, which is 1x1 + STBC + LDPC) which can quite often get mucked up with negotiation nonsense.
Comment 101 Cy Schubert freebsd_committer freebsd_triage 2022-06-19 18:41:18 UTC
Using my iwn(4) connected to one of my networks (now open) and reconfigured my laptop to remove wlan0 from lagg0:

wlan0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
	ether xx:xx:xx:xx:xx:xx
	inet6 fe80::120b:a9ff:fe1d:ce54%wlan0 prefixlen 64 scopeid 0x3
	inet 192.168.1.107 netmask 0xffffff00 broadcast 192.168.1.255
	groups: wlan
	ssid Golden_Pond-5G channel 161 (5805 MHz 11a ht/40-) bssid xx:xx:xx;xx;xx;xx
	regdomain FCC country US authmode OPEN privacy OFF txpower 23
	bmiss 10 mcastrate 6 mgmtrate 6 scanvalid 60 ampdulimit 64k
	ampdudensity 8 -amsdutx amsdurx shortgi -stbc -ldpc -uapsd wme
	roaming MANUAL bintval 200
	parent interface: iwn0
	media: IEEE 802.11 Wireless Ethernet MCS mode 11na
	status: associated
	nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL>

It connects and gets and IP. I'm using wpa_supplicant-devel from ports, which is newer than base. Can people try security/wpa_supplicant-devel? If it works we can merge it into base.

@adrain, I'll reach out to you offline regarding the NIC.
Comment 102 Cy Schubert freebsd_committer freebsd_triage 2022-06-19 19:06:28 UTC
I've been able to reproduce this problem with rtwn(4).

My iwn(4) works with open networks.

My ath(4) fails to scan anything, even in single user, suggesting that either the NIC no longer works or there is a driver problem. I'll try to verify the card after I restore my windows 10 partition.
Comment 103 rkoberman 2022-06-19 23:42:19 UTC
(In reply to Cy Schubert from comment #102)
Just to confirm, my urtwn fob also fails in the same manner with 13.1 and worked fine with 13.0. I'll have my urtwn with me when I do testing, almost certainly tomorrow. I'll certainly do the "ifconfig -v wlan0 list scan". 

Probably unrelated, but I just notices that my laptop fails to detect my 5G SSID. Works fine on my phone, though.
Comment 104 Cy Schubert freebsd_committer freebsd_triage 2022-06-20 01:20:34 UTC
Created attachment 234801 [details]
Revert driver_bsd.c from wpa 2.10 back to 2.9

Can you try this patch?

To apply,

cd /usr/src
git apply /tmp/driver_bsd.c-revert

or

cd /usr/src
patch -C -p1 < /tmp/driver_bsd.c-revert && patch -C -p1 < /tmp/driver_bsd.c-revert

Then,

cd /usr/src/usr.sbin/wpa
make obj
make depend
make include
make
make install

Then restart your interface.
Comment 105 Cy Schubert freebsd_committer freebsd_triage 2022-06-20 06:28:15 UTC
Created attachment 234807 [details]
A better patch

This patch will bring us closer.

This resolves connecting to an open unprotected 802.11g. For an 802.11n, please run the following test.

If wlan0 is up, ifconfig wlan0 down delete.

/usr/sbin/wpa_supplicant -dd -i wlan1 -c /etc/wpa_supplicant.conf.open -D bsd

Replace wlan1 with your wlan interface, probably wlan0. And, replace wpa_supplicant.conf.open with your wpa_supplicant.conf.

Notice that there is no -B flag. This is important. It will print out a lot of output while it attempts to associate. Let it try for a couple of minutes, then hit control-c and post the output here.
Comment 106 Jaskie 2022-06-20 08:05:23 UTC
(In reply to Cy Schubert from comment #105)

OK. I applied this patch (A better patch), compiled & installed wpa and tested. WiFi still not working. 

# ifconfig -v wlan0 list scan |grep WHU-STU
WHU-STU                           70:d9:31:0e:5e:20    1   54M  -83:-96   300 ES   SSID<WHU-STU> RATES<B2,B4,11,22,12,18,24,36> DSPARMS<1> COUNTRY<CN  1-13,20> ERP<0x2> XRATES<48,72,96,108> RRM_ENCAPS<46057100000000> HTCAP<cap 0x18c param 0x3 mcsset[0-15] extcap 0x0 txbf 0x0 antenna 0x0> HTINFO<ctl 1, 0,1,0,0 basicmcs[]> EXTCAP<7f080000080000000440> WME<qosinfo 0x6 BE[aifsn 3 cwmin 4 cwmax 10 txop 0] BK[aifsn 7 cwmin 4 cwmax 10 txop 0] VO[aifsn 2 cwmin 3 cwmax 4 txop 94] VI[aifsn 2 cwmin 2 cwmax 3 txop 47]> ATH<0x7fff>
WHU-STU                           70:d9:31:0e:14:a0    1   54M  -88:-96   300 ES   SSID<WHU-STU> RATES<B2,B4,11,22,12,18,24,36> DSPARMS<1> COUNTRY<CN  1-13,20> ERP<0x2> XRATES<48,72,96,108> RRM_ENCAPS<46057100000000> HTCAP<cap 0x18c param 0x3 mcsset[0-15] extcap 0x0 txbf 0x0 antenna 0x0> HTINFO<ctl 1, 0,1,0,0 basicmcs[]> EXTCAP<7f080000080000000440> WME<qosinfo 0x6 BE[aifsn 3 cwmin 4 cwmax 10 txop 0] BK[aifsn 7 cwmin 4 cwmax 10 txop 0] VO[aifsn 2 cwmin 3 cwmax 4 txop 94] VI[aifsn 2 cwmin 2 cwmax 3 txop 47]> ATH<0x7fff>
WHU-STU                           fc:b6:98:f5:b6:c0   13   54M  -87:-95   300 ES   SSID<WHU-STU> RATES<B2,B4,11,22,12,18,24,36> DSPARMS<13> TIM<050400010001> COUNTRY<CN  1-13,20> ERP<0x0> XRATES<48,72,96,108> RRM_ENCAPS<46057100000000> HTCAP<cap 0x18c param 0x3 mcsset[0-15] extcap 0x0 txbf 0x0 antenna 0x0> HTINFO<ctl 13, 8,0,0,0 basicmcs[]> EXTCAP<7f080000080000000440> WME<qosinfo 0x5 BE[aifsn 3 cwmin 4 cwmax 10 txop 0] BK[aifsn 7 cwmin 4 cwmax 10 txop 0] VO[aifsn 2 cwmin 3 cwmax 4 txop 94] VI[aifsn 2 cwmin 2 cwmax 3 txop 47]> ATH<0x7fff>
WHU-STU                           70:d9:31:0e:80:a0   13   54M  -85:-95   300 ES   SSID<WHU-STU> RATES<B2,B4,11,22,12,18,24,36> DSPARMS<13> TIM<050400010001> COUNTRY<CN  1-13,20> ERP<0x0> XRATES<48,72,96,108> RRM_ENCAPS<46057100000000> HTCAP<cap 0x18c param 0x3 mcsset[0-15] extcap 0x0 txbf 0x0 antenna 0x0> HTINFO<ctl 13, 8,0,0,0 basicmcs[]> EXTCAP<7f080000080000000440> WME<qosinfo 0x5 BE[aifsn 3 cwmin 4 cwmax 10 txop 0] BK[aifsn 7 cwmin 4 cwmax 10 txop 0] VO[aifsn 2 cwmin 3 cwmax 4 txop 94] VI[aifsn 2 cwmin 2 cwmax 3 txop 47]> ATH<0x7fff>
WHU-STU                           70:d9:31:0e:2c:00    5   54M  -78:-96   300 ES   SSID<WHU-STU> RATES<B2,B4,11,22,12,18,24,36> DSPARMS<5> TIM<050400010001> COUNTRY<CN  1-13,20> ERP<0x0> XRATES<48,72,96,108> RRM_ENCAPS<46057100000000> HTCAP<cap 0x18c param 0x3 mcsset[0-15] extcap 0x0 txbf 0x0 antenna 0x0> HTINFO<ctl 5, 0,5,0,0 basicmcs[]> EXTCAP<7f080000080000000440> WME<qosinfo 0x5 BE[aifsn 3 cwmin 4 cwmax 10 txop 0] BK[aifsn 7 cwmin 4 cwmax 10 txop 0] VO[aifsn 2 cwmin 3 cwmax 4 txop 94] VI[aifsn 2 cwmin 2 cwmax 3 txop 47]> ATH<0x7fff>
WHU-STU                           fc:b6:98:f5:cc:00    5   54M  -84:-96   300 ES   SSID<WHU-STU> RATES<B2,B4,11,22,12,18,24,36> DSPARMS<5> COUNTRY<CN  1-13,20> ERP<0x0> XRATES<48,72,96,108> RRM_ENCAPS<46057100000000> HTCAP<cap 0x18c param 0x3 mcsset[0-15] extcap 0x0 txbf 0x0 antenna 0x0> HTINFO<ctl 5, 0,5,0,0 basicmcs[]> EXTCAP<7f080000080000000440> WME<qosinfo 0x0 BE[aifsn 3 cwmin 4 cwmax 10 txop 0] BK[aifsn 7 cwmin 4 cwmax 10 txop 0] VO[aifsn 2 cwmin 3 cwmax 4 txop 94] VI[aifsn 2 cwmin 2 cwmax 3 txop 47]> ATH<0x7fff>
WHU-STU                           fc:b6:98:f6:01:a0    5   54M  -84:-96   300 ES   SSID<WHU-STU> RATES<B2,B4,11,22,12,18,24,36> DSPARMS<5> COUNTRY<CN  1-13,20> ERP<0x0> XRATES<48,72,96,108> RRM_ENCAPS<46057100000000> HTCAP<cap 0x18d param 0x3 mcsset[0-15] extcap 0x0 txbf 0x0 antenna 0x0> HTINFO<ctl 5, 0,1,0,0 basicmcs[]> EXTCAP<7f080000080000000440> WME<qosinfo 0x6 BE[aifsn 3 cwmin 4 cwmax 10 txop 0] BK[aifsn 7 cwmin 4 cwmax 10 txop 0] VO[aifsn 2 cwmin 3 cwmax 4 txop 94] VI[aifsn 2 cwmin 2 cwmax 3 txop 47]> ATH<0x7fff>
WHU-STU                           70:d9:31:0e:19:20    5   54M  -85:-96   300 ES   SSID<WHU-STU> RATES<B2,B4,11,22,12,18,24,36> DSPARMS<5> TIM<050400010001> COUNTRY<CN  1-13,20> ERP<0x0> XRATES<48,72,96,108> RRM_ENCAPS<46057100000000> HTCAP<cap 0x18c param 0x3 mcsset[0-15] extcap 0x0 txbf 0x0 antenna 0x0> HTINFO<ctl 5, 0,1,0,0 basicmcs[]> EXTCAP<7f080000080000000440> WME<qosinfo 0x3 BE[aifsn 3 cwmin 4 cwmax 10 txop 0] BK[aifsn 7 cwmin 4 cwmax 10 txop 0] VO[aifsn 2 cwmin 3 cwmax 4 txop 94] VI[aifsn 2 cwmin 2 cwmax 3 txop 47]> ATH<0x7fff>
WHU-STU                           fc:b6:98:f6:0a:20    9   54M  -81:-96   300 ES   SSID<WHU-STU> RATES<B2,B4,11,22,12,18,24,36> DSPARMS<9> COUNTRY<CN  1-13,20> ERP<0x0> XRATES<48,72,96,108> RRM_ENCAPS<46057100000000> HTCAP<cap 0x18c param 0x3 mcsset[0-15] extcap 0x0 txbf 0x0 antenna 0x0> HTINFO<ctl 9, 8,0,0,0 basicmcs[]> EXTCAP<7f080000080000000440> WME<qosinfo 0xc BE[aifsn 3 cwmin 4 cwmax 10 txop 0] BK[aifsn 7 cwmin 4 cwmax 10 txop 0] VO[aifsn 2 cwmin 3 cwmax 4 txop 94] VI[aifsn 2 cwmin 2 cwmax 3 txop 47]> ATH<0x7fff>
WHU-STU                           70:d9:31:12:9b:80    9   54M  -88:-96   300 ES   SSID<WHU-STU> RATES<B2,B4,11,22,12,18,24,36> DSPARMS<9> COUNTRY<CN  1-13,20> ERP<0x0> XRATES<48,72,96,108> RRM_ENCAPS<46057100000000> HTCAP<cap 0x18c param 0x3 mcsset[0-15] extcap 0x0 txbf 0x0 antenna 0x0> HTINFO<ctl 9, 8,0,0,0 basicmcs[]> EXTCAP<7f080000080000000440> WME<qosinfo 0x6 BE[aifsn 3 cwmin 4 cwmax 10 txop 0] BK[aifsn 7 cwmin 4 cwmax 10 txop 0] VO[aifsn 2 cwmin 3 cwmax 4 txop 94] VI[aifsn 2 cwmin 2 cwmax 3 txop 47]> ATH<0x7fff>
WHU-STU                           70:d9:31:03:06:c0    9   54M  -87:-96   300 ES   SSID<WHU-STU> RATES<B2,B4,11,22,12,18,24,36> DSPARMS<9> TIM<050400010001> COUNTRY<CN  1-13,20> ERP<0x0> XRATES<48,72,96,108> RRM_ENCAPS<46057100000000> HTCAP<cap 0x18c param 0x3 mcsset[0-15] extcap 0x0 txbf 0x0 antenna 0x0> HTINFO<ctl 9, 8,0,0,0 basicmcs[]> EXTCAP<7f080000080000000440> WME<qosinfo 0x0 BE[aifsn 3 cwmin 4 cwmax 10 txop 0] BK[aifsn 7 cwmin 4 cwmax 10 txop 0] VO[aifsn 2 cwmin 3 cwmax 4 txop 94] VI[aifsn 2 cwmin 2 cwmax 3 txop 47]> ATH<0x7fff>

Notice: bssid 70:d9:31:0e:2c:00 is usually the one I connect when using wpa 2.9.


/usr/sbin/wpa_supplicant -dd -i wlan0 -c /etc/wpa_supplicant.conf -D bsd output is too long to paste so I uploaded it as an attachment.
Thanks all of you for your effort!
Comment 107 Jaskie 2022-06-20 08:06:19 UTC
Created attachment 234808 [details]
wpa out

Output from /usr/sbin/wpa_supplicant -dd -i wlan0 -c /etc/wpa_supplicant.conf -D bsd
Comment 108 Cy Schubert freebsd_committer freebsd_triage 2022-06-20 14:02:39 UTC
Can you apply the latest patch first, then run the scan again. That patch was in the FreeBSD copy of 2.9 but not in 2.10.

To make it easier to capture the output, run this:

/usr/sbin/wpa_supplicant -dd -i wlan1 -c /etc/wpa_supplicant.conf.open -D bsd 2>&1 | tee output.file

Let it run for 90 seconds or more then hit ctrl-c.
Comment 109 commit-hook freebsd_committer freebsd_triage 2022-06-20 14:30:01 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=3b2956781048f300c081952150d515573a274620

commit 3b2956781048f300c081952150d515573a274620
Author:     Cy Schubert <cy@FreeBSD.org>
AuthorDate: 2022-06-20 14:21:55 +0000
Commit:     Cy Schubert <cy@FreeBSD.org>
CommitDate: 2022-06-20 14:25:35 +0000

    wpa: Restore missing patch

    In December after a failed MFV due to a now understood issue I had with
    git -- git aborts with extremely large MFV -- this patch was removed
    during the revert. Restore this patch.

    PR:             264238
    Fixes:          4b72b91a7132df1f77bbae194e1071ac621f1edb
    MFC after:      1 week

 contrib/wpa/src/drivers/driver_bsd.c | 24 ++++++++++++++++++++++++
 1 file changed, 24 insertions(+)
Comment 110 commit-hook freebsd_committer freebsd_triage 2022-06-20 14:30:02 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/ports/commit/?id=05a849eec9d949b3de32e464570cefbabcd64702

commit 05a849eec9d949b3de32e464570cefbabcd64702
Author:     Cy Schubert <cy@FreeBSD.org>
AuthorDate: 2022-06-20 14:25:42 +0000
Commit:     Cy Schubert <cy@FreeBSD.org>
CommitDate: 2022-06-20 14:29:09 +0000

    */*: Restore a missing wpa BSD driver patch

    These patches were removed to sync with base where in fact base was
    missing these patches and base should have been synced with the ports.

    PR:             264238
    Fixes:          b8477825c2dc42f6c595697a36f593c71f39fbad
                    c86f32d652eb9dd023049122d8ca37cb13ed07b6
    MFH:            2022Q2

 net/hostapd-devel/Makefile                         |  2 +-
 .../files/patch-src_drivers_driver__bsd.c          | 65 +++++++++++++++++++++-
 net/hostapd/Makefile                               |  2 +-
 net/hostapd/files/patch-src_drivers_driver__bsd.c  | 65 +++++++++++++++++++++-
 security/wpa_supplicant-devel/Makefile             |  2 +-
 .../files/patch-src_drivers_driver__bsd.c          | 65 +++++++++++++++++++++-
 security/wpa_supplicant/Makefile                   |  2 +-
 .../files/patch-src_drivers_driver__bsd.c          | 65 +++++++++++++++++++++-
 8 files changed, 256 insertions(+), 12 deletions(-)
Comment 111 commit-hook freebsd_committer freebsd_triage 2022-06-20 15:13:11 UTC
A commit in branch 2022Q2 references this bug:

URL: https://cgit.FreeBSD.org/ports/commit/?id=52efc7630680ad1445bf062b25d1f868e26e4d0d

commit 52efc7630680ad1445bf062b25d1f868e26e4d0d
Author:     Cy Schubert <cy@FreeBSD.org>
AuthorDate: 2022-06-20 14:25:42 +0000
Commit:     Cy Schubert <cy@FreeBSD.org>
CommitDate: 2022-06-20 15:12:01 +0000

    */*: Restore a missing wpa BSD driver patch

    These patches were removed to sync with base where in fact base was
    missing these patches and base should have been synced with the ports.

    PR:             264238
    Fixes:          b8477825c2dc42f6c595697a36f593c71f39fbad
                    c86f32d652eb9dd023049122d8ca37cb13ed07b6
    MFH:            2022Q2
    (cherry picked from commit 05a849eec9d949b3de32e464570cefbabcd64702)

 net/hostapd-devel/Makefile                         |  2 +-
 .../files/patch-src_drivers_driver__bsd.c          | 65 +++++++++++++++++++++-
 net/hostapd/Makefile                               |  2 +-
 net/hostapd/files/patch-src_drivers_driver__bsd.c  | 65 +++++++++++++++++++++-
 security/wpa_supplicant-devel/Makefile             |  2 +-
 .../files/patch-src_drivers_driver__bsd.c          | 65 +++++++++++++++++++++-
 security/wpa_supplicant/Makefile                   |  2 +-
 .../files/patch-src_drivers_driver__bsd.c          | 65 +++++++++++++++++++++-
 8 files changed, 256 insertions(+), 12 deletions(-)
Comment 112 commit-hook freebsd_committer freebsd_triage 2022-06-20 15:13:13 UTC
A commit in branch 2022Q2 references this bug:

URL: https://cgit.FreeBSD.org/ports/commit/?id=c82d2efea691ec4d8eac6a875eb0fe182106bf99

commit c82d2efea691ec4d8eac6a875eb0fe182106bf99
Author:     Cy Schubert <cy@FreeBSD.org>
AuthorDate: 2022-06-19 16:15:44 +0000
Commit:     Cy Schubert <cy@FreeBSD.org>
CommitDate: 2022-06-20 15:11:55 +0000

    */*: Bring back wpa_supplicant29 and hostapd29 as new ports

    The current wpa_supplicant and hostapd have an issue with AR9285.
    For the time being bring back wpa_supplicant 2.9 as
    security/wpa_supplicant29 and hostpd 2.9 as net/hostapd29 for those
    cases that have an issue with wpa_supplicant/hostpad2.10 (in base and
    in ports)

    PR:             264238
    (cherry picked from commit 7150a0c9b1014e445a8266c9080d0bf4738dcc9c)

 net/Makefile                                       |   1 +
 net/hostapd29/Makefile (new)                       |  46 +++
 net/hostapd29/distinfo (new)                       |   9 +
 net/hostapd29/files/config (new)                   | 316 ++++++++++++++++++
 net/hostapd29/files/hostapd.in (new)               |  39 +++
 .../patch-src-l2_packet-l2_packet_freebsd.c (new)  |  14 +
 net/hostapd29/files/patch-src_common_dhcp.h (new)  |  25 ++
 .../files/patch-src_drivers_driver__bsd.c (new)    |  60 ++++
 net/hostapd29/files/patch-src_utils_os.h (new)     |  17 +
 .../files/patch-src_utils_os__unix.c (new)         |  18 +
 .../files/patch-src_wps_wps__upnp.c (new)          |  20 ++
 net/hostapd29/pkg-descr (new)                      |  12 +
 net/hostapd29/pkg-message (new)                    |  10 +
 security/Makefile                                  |   1 +
 security/wpa_supplicant29/Makefile (new)           | 229 +++++++++++++
 security/wpa_supplicant29/distinfo (new)           |  11 +
 security/wpa_supplicant29/files/Packet32.c (new)   | 366 +++++++++++++++++++++
 security/wpa_supplicant29/files/Packet32.h (new)   |  65 ++++
 security/wpa_supplicant29/files/ntddndis.h (new)   |  32 ++
 .../files/patch-src_common_dhcp.h (new)            |  25 ++
 .../files/patch-src_drivers_driver__bsd.c (new)    |  48 +++
 .../files/patch-src_drivers_driver__ndis.c (new)   |  89 +++++
 ...atch-src_l2__packet_l2__packet__freebsd.c (new) |  12 +
 .../files/patch-src_radius_radius__client.c (new)  |  12 +
 .../files/patch-src_wps_wps__upnp.c (new)          |  34 ++
 .../files/patch-wpa__supplicant_Makefile (new)     |  17 +
 .../files/patch-wpa__supplicant_main.c (new)       |  33 ++
 .../patch-wpa__supplicant_wpa__supplicant.c (new)  |  16 +
 .../wpa_supplicant29/files/pkg-message.in (new)    |  11 +
 .../wpa_supplicant29/files/wpa_supplicant.in (new) |  54 +++
 security/wpa_supplicant29/pkg-descr (new)          |  14 +
 security/wpa_supplicant29/pkg-plist (new)          |   5 +
 32 files changed, 1661 insertions(+)
Comment 113 Jaskie 2022-06-20 15:28:40 UTC
(In reply to Cy Schubert from comment #108)

(In reply to Cy Schubert from comment #108)

Sorry I don't really understand what you mean by "apply the latest patch first". So I did the following test:

1. Download patch from #Comment104 as p1, patch from #Comment107 as p2
2. service netif stop
3. revert /usr/src to release-13.1.0
4. git apply p1
5. compile and test wpa, output wpa_out1.txt (uploaded as attachment)
6. git apply p2 (if I understand it right, this means p2 + p1)
7. compile wpa

But I didn't succeed at step 7, wpa failed to compile at make:

ld: error: undefined symbol: bsd_get_iface_flags
>>> referenced by driver_bsd.c:1137 (/usr/src/contrib/wpa/src/drivers/driver_bsd.c:1137)
>>>               driver_bsd.o:(wpa_driver_bsd_associate)
cc: error: linker command failed with exit code 1 (use -v to see invocation)
*** Error code 1

Stop.
make[1]: stopped in /usr/src/usr.sbin/wpa/wpa_supplicant
*** Error code 1
Comment 114 Jaskie 2022-06-20 15:29:21 UTC
Created attachment 234816 [details]
wpa out p1
Comment 115 rkoberman 2022-06-21 19:58:30 UTC
(In reply to Cy Schubert from comment #105)
I'm about to rebuild with the latest wps_supplicant. Just to clarify, does this refer to connecting to an AP supporting 11n or the supplicant attempting to associate at 11n (which I assume includes 11ng.) I ask because iwlwifi(4) has no 11n capability at this time.

Also, jaskie seems to have already provided the information I collected yesterday, but I do want to confirm that the issues does apply to iwlwifi(4) and rtwm(4). Both work with 2.9, but neither work with the base supplicant.

I will note that the iwlwifi seemed to crash a lot when I tried using it with the base system supplicant. rtwn had no such issues.

Thanks for your everyone, especially cy, for all of their work on this very annoying bug!
Comment 116 Cy Schubert freebsd_committer freebsd_triage 2022-06-21 20:17:32 UTC
The difference between an open AP that my rtwn(4) can associate to and one it cannot is:

associates;
wpa_driver_bsd_associate: ssid 'KQNGN3' wpa ie len 0 pairwise 1 group 1 key mgmt 4

Does not associate:
wpa_driver_bsd_associate: ssid 'Golden_Pond' wpa ie len 13 pairwise 1 group 1 key mgmt 4

The Information Element appears to cause it to use WPA on an open network.
Comment 117 Cy Schubert freebsd_committer freebsd_triage 2022-06-21 20:23:59 UTC
(In reply to rkoberman from comment #115)

I'd be interested in seeing this line of output from wpa_supplicant -dd

wpa_driver_bsd_associate: ssid 'Golden_Pond' wpa ie len 13 pairwise 1 group 1 key mgmt 4

Seems that an IE supplied by the AP causes wpa_supplicant to attempt to associate using WPA against an open AP, even when the config file says key_mgmt=NONE.
Comment 118 Cy Schubert freebsd_committer freebsd_triage 2022-06-21 20:53:15 UTC
Under wpa_supplicant 2.9 the AP in question prints,

wpa_driver_bsd_associate: ssid 'Golden_Pond' wpa ie len 0 pairwise 1 group 1 key mgmt 4

Notice that there is no IE.

2.9 ignores the IE whereas 2.10 does not.
Comment 119 Cy Schubert freebsd_committer freebsd_triage 2022-06-21 21:21:07 UTC
So the plot thickens.

Golden_Pond, is an open guest network I created on my DMZ AP. Removing the guest SSID and opening up the DMZ AP entirely resolves the problem. So, what does this mean?

An open network will not have an IE. However an AP which has a primary WPA SSID and a secondary (like a guest SSID) which is open will broadcast a WPA IE. And, wpa_suppliant uses that to communicate its WPA IE. wpa_supplicant takes that to mean it's WPA protected.

So the question is, what in wpa_supplicant changed tfor an open network with key_mgmt=NONE for it to override key_mgmt?

I will post this to the hostap mailing list.
Comment 120 Adrian Chadd freebsd_committer freebsd_triage 2022-06-21 21:25:26 UTC
Cy, does it do that on freebsd as an AP? that's ... kinda wrong?
Comment 121 Cy Schubert freebsd_committer freebsd_triage 2022-06-21 21:43:32 UTC
(In reply to Adrian Chadd from comment #120)

I haven't tried FreeBSD as an AP yet.

The "bug" is on a Netgear router (used as a router in my DMZ). It normally has one SSID (my wife's SSID). Adding a guest network to it causes the problem.

Looking at WHU-STU, this is not the case.

wlan0: 9: 70:d9:31:0e:2c:00 ssid='WHU-STU' wpa_ie_len=0 rsn_ie_len=0 caps=0x21 l
evel=-70 freq=2432

In other words I have not been able to reproduce this problem here because my testing has exposed a bug in this Netgear router.
Comment 122 Cy Schubert freebsd_committer freebsd_triage 2022-06-21 22:27:16 UTC
(In reply to Adrian Chadd from comment #120)

Using it with an open FreeBSD hostapd AP works perfectly. wpa_ie is zero.
Comment 123 Cy Schubert freebsd_committer freebsd_triage 2022-06-21 23:32:09 UTC
Created attachment 234848 [details]
Remove upstream cc79eb725f7b564269619ce4a64be2a80927d392

Can everyone please try this patch. If you're running 13.1-RELEASE you will also need to include the patch at https://bugs.freebsd.org/bugzilla/attachment.cgi?id=234807&action=edit.
Comment 124 Jaskie 2022-06-22 14:42:58 UTC
(In reply to Cy Schubert from comment #123)

Tested. WiFi didn't work. /usr/sbin/wpa_supplicant -dd -i wlan1 -c /etc/wpa_supplicant.conf.open -D bsd 2>&1 | tee output.file output (~3min) uploaded.
Comment 125 Jaskie 2022-06-22 14:43:43 UTC
Created attachment 234857 [details]
output from wap 2.10 patched (2 patches)
Comment 126 Glen Barber freebsd_committer freebsd_triage 2022-06-22 14:52:46 UTC
EN candidate, once the problem is confirmed fixed.

Also, fix the re@ CC and remove myself, as I implicitly receive those (and am actively watching this PR).
Comment 127 Cy Schubert freebsd_committer freebsd_triage 2022-06-22 15:08:44 UTC
(In reply to Jaskie from comment #125)

Thank you. A different error message to work on now. (Sadly, not being able to reproduce this problem here is a disadvantage.)

Hypothesis: I suspect the problem is multiple same SSIDs on the same network, like a hotel or school. I cannot reproduce this with two and three APs here with the same SSID. Multiple independent routers and access points, like two independent routers and a FreeBSD hostapd advertising the same SSID has not yet exhibited the problem here.

Is anyone here with this same problem using WPA at a hotel or school? Or does this affect only open unprotected networks?
Comment 128 Cy Schubert freebsd_committer freebsd_triage 2022-06-23 03:04:25 UTC
Can everyone here please confirm security/wpa_supplicant also does not work.

The port has disabled P2P support by default while base has it enabled.
Comment 129 Adrian Chadd freebsd_committer freebsd_triage 2022-06-23 03:19:43 UTC
heh, we shouldn't have p2p enabled until we have support for it. :-)
Comment 130 Cy Schubert freebsd_committer freebsd_triage 2022-06-23 03:50:39 UTC
Yeah, it's not the issue anyway. Read my next post.
Comment 131 Cy Schubert freebsd_committer freebsd_triage 2022-06-23 03:57:07 UTC
Hi Jackie,

Reviewing one of your previous outputs (wpa_out_p1) we see:

wlan0: Automatic auth_alg selection: 0x1
No supported operating classes IE to add
wlan0: Trying to associate with 70:d9:31:0e:2c:00 (SSID='WHU-STU' freq=2432 MHz)
wlan0: Cancelling scan request
wlan0: State: SCANNING -> ASSOCIATING
Limit connection to BSSID 70:d9:31:0e:2c:00 freq=2432 MHz based on scan results
(bssid_set=0 wps=0)
wpa_driver_bsd_associate: ssid 'WHU-STU' wpa ie len 13 pairwise 1 group 1 key mg
mt 4
                                         ^^^^^^^^^^^^^
Above is the problem. This is the same problem my buggy Netgear router exhibited with an open guest network when the primary network was configured for WPA-PSK.

wpa_driver_bsd_set_drop_unencrypted: enabled=0
bsd_set_opt_ie: set WPA+RSN ie (len 13)

Again, we see an IE of 13 bytes telling wpa_supplicant to use the WPA protocol with RSN encryption.

wpa_driver_bsd_associate: set PRIVACY 1

And of course privacy is enabled. Same thing I saw with the buggy Netgear.

wlan0: Setting authentication timeout: 10 sec 0 usec

@adrian, do you have any thoughts about this? Could the university be hosting another AP for faculty and staff off the same hardware?
Comment 132 commit-hook freebsd_committer freebsd_triage 2022-06-23 03:58:24 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=3e8eb5c7f4909209c042403ddee340b2ee7003a5

commit 3e8eb5c7f4909209c042403ddee340b2ee7003a5
Author:     Cy Schubert <cy@FreeBSD.org>
AuthorDate: 2022-06-23 03:51:27 +0000
Commit:     Cy Schubert <cy@FreeBSD.org>
CommitDate: 2022-06-23 03:57:27 +0000

    wpa: Disable P2P in WPS

    Disable P2P in WPS as it is not supported by FreeBSD. Also, it is not
    enabled in wpa_supplicant so the WPS P2P code is redundant.

    PR:             264238
    Reported by:    adrian
    MFC after:      3 days

 usr.sbin/wpa/src/wps/Makefile | 2 --
 1 file changed, 2 deletions(-)
Comment 133 Adrian Chadd freebsd_committer freebsd_triage 2022-06-23 04:08:02 UTC
hm, so if the WPA IEs are /actually/ making it into the TXed frames and it's not a net80211 / driver bug somehow .... ew.

Yes, it's common to have open + WPA VAPs. Heck, I do it at home sometimes for ye olde equipment on a separate VLAN.
Comment 134 Cy Schubert freebsd_committer freebsd_triage 2022-06-23 04:16:43 UTC
Yes, but should it add those frames to the open VAP too?

Can you check your equipment for the same "feature?"
Comment 135 Adrian Chadd freebsd_committer freebsd_triage 2022-06-23 04:20:22 UTC
a) it doesn't on a tplink
b) it should not, no
Comment 136 Adrian Chadd freebsd_committer freebsd_triage 2022-06-23 04:21:06 UTC
oh crazy q - are the VAP MAC addresses different? Between the WPA and open VAPs?
Comment 137 Cy Schubert freebsd_committer freebsd_triage 2022-06-23 05:22:33 UTC
WRT one of my Netgear APs, no, they do not share the same MAC. However the 2.4 GHz and 5 GHz secondary VAPs share the same channel as their primary VAPs. Makes sense as the device has two radios.

The open guest VAPs broadcast an IE while the primary VAPs do not when configured open.
Comment 138 Jaskie 2022-06-23 06:22:07 UTC
(In reply to Cy Schubert from comment #131)

I don't understand the tech stuff and code behind this. But I can confirm that there is only one WiFi hosted. We have different WiFi setup at different workplace on campus, but we always share the same WiFi name, students and stuff, we connect to the same WiFi using our own user id and password. The one I am having trouble with is in our student dormitory, hence the name has "STU" in it because it's intended for students use only in our dormitory.
Comment 139 J.R. Oldroyd 2022-06-23 09:37:17 UTC
Would a wireshark or tcpdump with radiotap on to view the beacon management frames be of any use here?
Comment 140 Cy Schubert freebsd_committer freebsd_triage 2022-06-23 13:11:36 UTC
No, tcpdump will dump no packets, even in monitor mode, during scan and association.
Comment 141 J.R. Oldroyd 2022-06-23 15:23:51 UTC
(In reply to Cy Schubert from comment #140)

Won't the beacons be the same both before and after association as far as the WPA IEs are concerned?

So dump the radiotap after association using the working wpa-supplicant and have a look where these IEs are.  At least it would show what bssids the ap is sending and their security setup.
Comment 142 Adrian Chadd freebsd_committer freebsd_triage 2022-06-23 15:41:42 UTC
So, the way this works is:

* during scan the net80211 stack just stores the beacons itself as scan results
* then it passes them up to wpa_supplicant via an ioctl
* then wpa_s uses the initial scan results to issue a "join this BSS as a station" ioctl
* <--- this is where we know the IEs are wrong
* then net80211 will do the join request, either via directly asking the wifi firmware to do the join, or by crafting auth/assoc frames and sending them to the AP to do the joining

Now, the IE contents between beacon, auth and assoc can be different as different subsets of info are required in each. But the only info used by wpa_s when joining a network is the contents of the scan result entry.

Ok, so!

Some NICs, like the intel NICs, don't send up beacon frames. Instead, they actually do the scanning /in firmware/, and then they send up scan results. That way the firmware can do stuff like background scanning without the driver/stack needing to buffer traffic - it's all done in firmware.

Other NICs, like rtwn, ath, etc - they're fully softmac NICs, and everything is done in the driver/stack. When you do a scan, the driver/stack will change channels, configure to receive beacon frames from all MACs, and listen for a bit.

Ok, so given that!

* For iwn, iwm, etc - they're the smart ones, tcpdump won't get the beacons. You only get what the firmware returns.
* For ath, rtwn, etc - they're softmac, tcpdump in 80211 monitor mode will see beacons.

ok, so!! :-P

* For AR9285, rtwn, etc you can run tcpdump -ni wlan0 -y IEEE802_11_RADIO during a scan, and it SHOULD show the beacons coming in as it does a scan. You can try it, see what happens.
* For iwn, iwm, etc - tcpdump won't help, and we'll have to use driver debugging to dump the scan results messages as they come in to see what's in there.

(For this reason I keep atheros NICs around for doing monitor mode sniffing, since sometimes firmware/drivers on smart / fullmac devices get things subtly weirdly wrong. :-)
Comment 143 J.R. Oldroyd 2022-06-23 15:48:09 UTC
(In reply to Adrian Chadd from comment #142)

Very useful info, thanks.

Except, I would point out that at the moment, I am on:
  iwm0: <Intel(R) Dual Band Wireless AC 8265>
and I can dump my beacons after association using tcpdump.
Comment 144 Adrian Chadd freebsd_committer freebsd_triage 2022-06-23 15:51:37 UTC
yup, once you've associated you can dump beacons on your current channel. :-) You just can't get a dump of all the beacons on all the channels during scan!
Comment 145 Bjoern A. Zeeb freebsd_committer freebsd_triage 2022-06-23 16:06:48 UTC
(In reply to Adrian Chadd from comment #144)

While diverting from the actual topic;  on Intel that still depends on what you tell the firmware even after assoc (also depends on PS and other things).  FW/driver way too clever ... and as it turns out sometimes stumbles over its own feet.

But yes -- if you want to get radiotap with those, use a 3rd device the listen to what you want/need which can get it to you unfiltered.
Comment 146 Cy Schubert freebsd_committer freebsd_triage 2022-06-24 00:01:56 UTC
Created attachment 234903 [details]
Proof of Concept solution

Please try this proof of concept solution. It's a bit of a hack but if it works (it works on my buggy Netgear) @adrian, @bz and I can discuss how to make this permanent).

Also make sure to apply https://bugs.freebsd.org/bugzilla/attachment.cgi?id=234807&action=edit along with it.

Let me know if this works for you too.

As I said, it's a bit of a hack until the root cause can be found but as a proof of concept, if it works for you as it did for me, we'll know we're heading in the right direction.
Comment 147 Cy Schubert freebsd_committer freebsd_triage 2022-06-24 12:23:39 UTC
I will put my work on hold until the POC solution has been tested here.
Comment 148 Jaskie 2022-06-24 14:48:04 UTC
(In reply to Cy Schubert from comment #146)

FINALLY, confirmed, wpa_supplicant worked after applying the two patches (attachment 234807 [details] and 234903). 
I don't know if you still need it, but I got tcpdump beacons from wpa 2.9 and 2.10 (the patched working version) and uploaded to attachment.

Thanks for all of your hard work and patience. This has been the most pleasant bug report experience for me, and a perfect presentation of why FreeBSD has been such a great OS to use. Thanks!
Comment 149 Jaskie 2022-06-24 14:50:10 UTC
Created attachment 234915 [details]
tcpdump beacons from wpa

tcpdump beacons from wpa 2.9 and patched 2.10 (patches from attachment 234807 [details] & 234903)
Comment 150 Cy Schubert freebsd_committer freebsd_triage 2022-06-24 14:58:25 UTC
Thank you. I'll take a look at them.
Comment 151 Cy Schubert freebsd_committer freebsd_triage 2022-06-24 15:27:10 UTC
Even though the patch is hackish, I may commit it in the interim to resolve this ticket while working on a long term solution. The code this patch touches is 14 years old so, obviously it is not the cause of the regression. The regression must be elsewhere. But if it resolves the immediate problem it's better than brokenness.

As to what upstream patch caused the regression, it is still under investigation. Ideally the problematic upstream commit will need to be found and discussed with the folks at w1.fi. I don't know if this problem only affects FreeBSD or if other BSDs or Linux are also affected by it.

One can reproduce this on 14-CURRENT with a secondary unproteced open VAP on Netgear WNDR3700v3 with a primary VAP configured for WPA-PSK.
Comment 152 Cy Schubert freebsd_committer freebsd_triage 2022-06-24 21:38:54 UTC
This is definitely a wpa_supplicant 2.10 problem. Testing this on Fedora 36 results in the same error on an open secondary VAP when the primary VAP is configured for WPA.
Comment 153 groenveld 2022-06-25 02:03:01 UTC
Short of a patch, Neel Chauhan's hack in bug #262183 to disable WPA supplication
works for me on current:
# sysrc ifconfig_wlan0="ssid OPENSSID DHCP"
# service netif restart
John
groenveld@acm.org
Comment 154 Cy Schubert freebsd_committer freebsd_triage 2022-06-25 04:14:29 UTC
(In reply to groenveld from comment #153)

Did you try the Proof of Concept Solution patch, at https://bugs.freebsd.org/bugzilla/attachment.cgi?id=234903 and the prerequisite patch at https://bugs.freebsd.org/bugzilla/attachment.cgi?id=234807 ?
Comment 155 Cy Schubert freebsd_committer freebsd_triage 2022-06-25 14:45:11 UTC
In order to provide relief to 13.1-RELEASE users we should implement the temporary workaround (POC solution). A review has been submitted at https://reviews.freebsd.org/D35596.
Comment 156 Cy Schubert freebsd_committer freebsd_triage 2022-06-25 14:46:50 UTC
This also affects the security/wpa_supplicant and security/wpa_supplicant-devel ports.
Comment 157 Jaskie 2022-06-25 14:51:41 UTC
(In reply to Cy Schubert from comment #152)


I test on another laptop running Debian sid with wpa 2.10 (more here: https://tracker.debian.org/pkg/wpa) and there is no problem. But the wireless card is different too:

% lspci -v |grep Wireless -A5                                                                                                           @22:45
02:00.0 Network controller: Intel Corporation Dual Band Wireless-AC 3165 Plus Bluetooth (rev 99)
        Subsystem: Intel Corporation Dual Band Wireless-AC 3165
        Flags: bus master, fast devsel, latency 0, IRQ 127
        Memory at d1100000 (64-bit, non-prefetchable) [size=8K]
        Capabilities: <access denied>
        Kernel driver in use: iwlwifi
        Kernel modules: iwlwifi

The laptop with ath card (the one I encountered this PR) is not running Linux so I cannot test.
Comment 158 commit-hook freebsd_committer freebsd_triage 2022-06-26 00:45:54 UTC
A commit in branch stable/13 references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=0fe8af4b2ae8b926e400f4b6eb6dbab43a91b21c

commit 0fe8af4b2ae8b926e400f4b6eb6dbab43a91b21c
Author:     Cy Schubert <cy@FreeBSD.org>
AuthorDate: 2022-06-23 03:51:27 +0000
Commit:     Cy Schubert <cy@FreeBSD.org>
CommitDate: 2022-06-26 00:44:47 +0000

    wpa: Disable P2P in WPS

    Disable P2P in WPS as it is not supported by FreeBSD. Also, it is not
    enabled in wpa_supplicant so the WPS P2P code is redundant.

    PR:             264238
    Reported by:    adrian

    (cherry picked from commit 3e8eb5c7f4909209c042403ddee340b2ee7003a5)

 usr.sbin/wpa/src/wps/Makefile | 2 --
 1 file changed, 2 deletions(-)
Comment 159 commit-hook freebsd_committer freebsd_triage 2022-06-26 00:45:56 UTC
A commit in branch stable/13 references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=1ecfe31e4272aad957eca983b48c783658a5e608

commit 1ecfe31e4272aad957eca983b48c783658a5e608
Author:     Cy Schubert <cy@FreeBSD.org>
AuthorDate: 2022-06-20 14:21:55 +0000
Commit:     Cy Schubert <cy@FreeBSD.org>
CommitDate: 2022-06-26 00:44:37 +0000

    wpa: Restore missing patch

    In December after a failed MFV due to a now understood issue I had with
    git -- git aborts with extremely large MFV -- this patch was removed
    during the revert. Restore this patch.

    PR:             264238
    Fixes:          4b72b91a7132df1f77bbae194e1071ac621f1edb

    (cherry picked from commit 3b2956781048f300c081952150d515573a274620)

 contrib/wpa/src/drivers/driver_bsd.c | 24 ++++++++++++++++++++++++
 1 file changed, 24 insertions(+)
Comment 160 commit-hook freebsd_committer freebsd_triage 2022-06-26 00:47:57 UTC
A commit in branch stable/12 references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=8784898e9a6d18c1d9eda42c207fda0092f84233

commit 8784898e9a6d18c1d9eda42c207fda0092f84233
Author:     Cy Schubert <cy@FreeBSD.org>
AuthorDate: 2022-06-23 03:51:27 +0000
Commit:     Cy Schubert <cy@FreeBSD.org>
CommitDate: 2022-06-26 00:47:11 +0000

    wpa: Disable P2P in WPS

    Disable P2P in WPS as it is not supported by FreeBSD. Also, it is not
    enabled in wpa_supplicant so the WPS P2P code is redundant.

    PR:             264238
    Reported by:    adrian

    (cherry picked from commit 3e8eb5c7f4909209c042403ddee340b2ee7003a5)

 usr.sbin/wpa/src/wps/Makefile | 2 --
 1 file changed, 2 deletions(-)
Comment 161 commit-hook freebsd_committer freebsd_triage 2022-06-26 00:47:59 UTC
A commit in branch stable/12 references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=7f8db05c3e87eab16e6f107b6ce94f16e4b1e75a

commit 7f8db05c3e87eab16e6f107b6ce94f16e4b1e75a
Author:     Cy Schubert <cy@FreeBSD.org>
AuthorDate: 2022-06-20 14:21:55 +0000
Commit:     Cy Schubert <cy@FreeBSD.org>
CommitDate: 2022-06-26 00:47:05 +0000

    wpa: Restore missing patch

    In December after a failed MFV due to a now understood issue I had with
    git -- git aborts with extremely large MFV -- this patch was removed
    during the revert. Restore this patch.

    PR:             264238
    Fixes:          4b72b91a7132df1f77bbae194e1071ac621f1edb

    (cherry picked from commit 3b2956781048f300c081952150d515573a274620)

 contrib/wpa/src/drivers/driver_bsd.c | 24 ++++++++++++++++++++++++
 1 file changed, 24 insertions(+)
Comment 162 Jaskie 2022-06-27 09:27:41 UTC
Do I now wait for the commits to get into pkg repository? How do I track if it's made into pkg repository?
Comment 163 Cy Schubert freebsd_committer freebsd_triage 2022-06-27 14:25:16 UTC
(In reply to Jaskie from comment #162)

There are two parts to this. wpa_supplicant exists in the base FreeBSD and it exists in ports.

For base, I will add the PR to the commit log message. You will see the commit to main here, plus you will see the MFC (merge from current) commits here too. Release Engineer said they will publish an EN. When they do you can freebsd-update.

For ports, there are three separate ports: wpa_supplicant, wpa_supplicant-devel, and wpa_supplicant29. The patch will be applied to wpa_supplicant and wpa_supplicant-devel when I apply the patch to freebsd/main. The patch will be merged to quarterly or more likely 2022Q3 will be cut by the ports management team on July 1 negating my need to merge from ports main to 2022Q2.

We need to wait for the code review to complete. Also remember, this patch is a temporary hack. It works by circumventing the whole issue but it does not address the cause.

The fact that I am able to reproduce this problem on the same secondary VAP using FreeBSD 14 and Fedora 36 but you are not able to do so with Debian suggests either that Debian has addressed the flaw or there may be a driver issue at play. Or, something else. With this in mind, I am considering building wpa_supplicant with -O0 to rule out llvm.
Comment 164 J.R. Oldroyd 2022-07-01 13:06:40 UTC
This may have something to do with the problem...

diff --git a/contrib/wpa/wpa_supplicant/wpa_supplicant.c b/contrib/wpa/wpa_supplicant/wpa_supplicant.c
index cf990f5a8f8..ba2dd470071 100644
--- a/contrib/wpa/wpa_supplicant/wpa_supplicant.c
+++ b/contrib/wpa/wpa_supplicant/wpa_supplicant.c
@@ -3141,7 +3141,7 @@ static u8 * wpas_populate_assoc_ies(
        else
                wpa_drv_get_ext_capa(wpa_s, WPA_IF_STATION);
 
-       if (!bss || wpa_bss_get_ie(bss, WLAN_EID_EXT_CAPAB)) {
+       if (!bss || wpa_bss_get_ie_ext(bss, WLAN_EID_EXT_CAPAB)) {
                u8 ext_capab[18];
                int ext_capab_len;
                ext_capab_len = wpas_build_ext_capab(wpa_s, ext_capab,
Comment 165 Cy Schubert freebsd_committer freebsd_triage 2022-07-01 14:07:57 UTC
(In reply to J.R. Oldroyd from comment #164)

I doubt it because this code was in wpa_supplicant since 2013:


commit 8b3b803ab9fe69650da7e3b2ee9e44f0f054ee0a
Author:     Arif Hussain <c_arifh@qca.qualcomm.com>
AuthorDate: Wed Oct 2 07:38:35 2013 -0700
Commit:     Jouni Malinen <j@w1.fi>
CommitDate: Wed Oct 2 08:09:05 2013 -0700

    Include Extended Capabilities element based on scan results
    
    Add Extended Capabilities element to association request only if the AP
    included this element in Beacon/Probe Response frames. This is a
    workaround to address interoperability issues with some older APs that
    do not seem to be able to handle Extended Capabilities element in
    (Re)Association Request frames.
    
    Signed-hostap: Jouni Malinen <jouni@qca.qualcomm.com>

diff --git a/wpa_supplicant/wpa_supplicant.c b/wpa_supplicant/wpa_supplicant.c
index eab1c39a4..99e48eb6b 100644
--- a/wpa_supplicant/wpa_supplicant.c
+++ b/wpa_supplicant/wpa_supplicant.c
@@ -1278,8 +1278,6 @@ void wpa_supplicant_associate(struct wpa_supplicant *wpa_s,
        int wep_keys_set = 0;
        int assoc_failed = 0;
        struct wpa_ssid *old_ssid;
-   u8 ext_capab[10];
-   int ext_capab_len;
 #ifdef CONFIG_HT_OVERRIDES
        struct ieee80211_ht_capabilities htcaps;
        struct ieee80211_ht_capabilities htcaps_mask;
@@ -1491,15 +1489,27 @@ void wpa_supplicant_associate(struct wpa_supplicant *wpa_s,
        }
 #endif /* CONFIG_HS20 */
 
-   ext_capab_len = wpas_build_ext_capab(wpa_s, ext_capab);
-   if (ext_capab_len > 0) {
-           u8 *pos = wpa_ie;
-           if (wpa_ie_len > 0 && pos[0] == WLAN_EID_RSN)
-                   pos += 2 + pos[1];
-           os_memmove(pos + ext_capab_len, pos,
-                      wpa_ie_len - (pos - wpa_ie));
-           wpa_ie_len += ext_capab_len;
-           os_memcpy(pos, ext_capab, ext_capab_len);
+ /*
+  * Workaround: Add Extended Capabilities element only if the AP
+  * included this element in Beacon/Probe Response frames. Some older
+  * APs seem to have interoperability issues if this element is
+  * included, so while the standard may require us to include the
+  * element in all cases, it is justifiable to skip it to avoid
+  * interoperability issues.
+  */
+ if (!bss || wpa_bss_get_ie(bss, WLAN_EID_EXT_CAPAB)) {
+         u8 ext_capab[10];
+         int ext_capab_len;
+         ext_capab_len = wpas_build_ext_capab(wpa_s, ext_capab);
+         if (ext_capab_len > 0) {
+                 u8 *pos = wpa_ie;
+                 if (wpa_ie_len > 0 && pos[0] == WLAN_EID_RSN)
+                         pos += 2 + pos[1];
+                 os_memmove(pos + ext_capab_len, pos,
+                            wpa_ie_len - (pos - wpa_ie));
+                 wpa_ie_len += ext_capab_len;
+                 os_memcpy(pos, ext_capab, ext_capab_len);
+         }
        }
 
        wpa_clear_keys(wpa_s, bss ? bss->bssid : NULL);



(Though the hypothesis is still worth investigating.)
Comment 166 J.R. Oldroyd 2022-07-01 14:18:53 UTC
I saw that the code was there for ages.

I should perhaps have said that I am now at a place where there is a cable modem with a primary SSID with WPA/RSN and a secondary SSID which is OPEN.  So I have been able to reproduce the problem myself now too.

Some printf instrumentation showed that the IE is being added in that "Workaround" section.

The comments for wpa_bss_get_ie() and wpa_bss_get_ie_ext() show that the _ext variant is to be used for WLAN_EID_EXT_*.

Perhaps this code was never reached in earlier versions and some other change now allows it to be executed?  That still needs investigating.

The patch does appear to fix the problem when applied to the stable/13 code with no other patches.
Comment 167 J.R. Oldroyd 2022-07-01 14:36:28 UTC
Ziggo-gastF69A115                 92:5c:34:09:73:bb    1   54M  -83:-96   100 E    APCHANREP APCHANREP HTCAP WME BSSLOAD
ZiggoF69A115                      d4:6e:0e:34:09:d4    6   54M  -57:-96   100 EPS  HTCAP WME ATH RSN WPA
Comment 168 Cy Schubert freebsd_committer freebsd_triage 2022-07-01 14:37:14 UTC
(In reply to J.R. Oldroyd from comment #166)

Agreed, still worth looking at. I'll look at that tonight or tomorrow. It certainly would be a more elegant fix than the hackish workaround.
Comment 169 Cy Schubert freebsd_committer freebsd_triage 2022-07-02 05:23:07 UTC
Created attachment 235029 [details]
Try ext IE first

(In reply to J.R. Oldroyd from comment #164)

Having had a chance to study this tonight, looks like you've found the bug. The difference between wpa_bss_get_ie() and wpa_bss_get_ie_ext() is that the latter returns and extended IE only if:

                if (element->id == WLAN_EID_EXTENSION &&                \
                    element->datalen > 0 &&                             \
                    element->data[0] == (extid))

WLAN_EID_EXT_CAPAB can reside in its own element or within WLAN_EID_EXTENSION. The attached patch is more appropriate.

Looking for WLAN_EID_EXT_CAPAB within WLAN_EID_EXT_CAPAB *only* will probably cause other bugs. We look for the extended capability in the EID extension first otherwise continue as before. Addresses the problem more elegantly.

Thanks for the excellent sleuthing and pointing to an elegant solution!
Comment 170 J.R. Oldroyd 2022-07-02 07:56:32 UTC
(In reply to Cy Schubert from comment #169)

Sorry, Cy, but your refined patch in attachment "Try ext IE first" does not work.  I have just tested it to confirm.

You call wpa_bss_get_ie_ext(), which correctly returns nothing, but then you proceed to call wpa_bss_get_ie() after and that still incorrectly returns the WPA IE which causes the driver to still set WPA when it should not.

I think we just need the patch from comment #164 which calls only wpa_bss_get_ie_ext().
Comment 171 Cy Schubert freebsd_committer freebsd_triage 2022-07-02 08:43:03 UTC
(In reply to J.R. Oldroyd from comment #170)

Thanks for testing. It worked properly here, telling us there are differences between your AP and mine here.
Comment 172 Cy Schubert freebsd_committer freebsd_triage 2022-07-02 08:52:24 UTC
Created attachment 235030 [details]
Uese ext IE only

(In reply to J.R. Oldroyd from comment #170)

OK, this confirms differences between your AP and mine, This patch implemements your patch and will be the one to be committed. Please test.
Comment 173 J.R. Oldroyd 2022-07-02 09:05:52 UTC
Yes, that works since it is the same as comment #164.

Further clarification...

In your earlier revised patch, the first test:
    if (!bss)
probably should have been:
    if (bss)

because the original code was:
    if (!bss || wpa_bss_get_ie(...))

so the get_ie function is only called if bss is set.

However, this does not make it work since, as I said, wpa_bss_get_ie() is still called if wpa_bss_get_ie_ext() failed.
Comment 174 Bjoern A. Zeeb freebsd_committer freebsd_triage 2022-07-02 12:44:29 UTC
(In reply to Cy Schubert from comment #172)

I believe the patch is wrong and is simply masking the real problem.

WLAN_EID_EXT_CAPAB is named WLAN_EID_EXT_* but like WLAN_EID_EXT_SUPP_RATES and WLAN_EID_EXT_CHANSWITCH_ANN not actually an Element ID extension (not EID 255).

I believe you'll always get NULL back from that call now as there is no {EID 255, Element ID Extension 127} (defined yet).
Comment 175 J.R. Oldroyd 2022-07-02 13:51:58 UTC
(In reply to Bjoern A. Zeeb from comment #174)

Hmm.

I am looking at IEEE 802.11-2016 in section 9.4.2.1 (pg 784).

  Element ID 127 is defined as:
    Extended Capabilities (see 9.4.2.27)

  Element ID 255 is defined as:
    Reserved for elements using the Element ID Extension field

Section 9.4.2.27 (pg 890) describes the Extended Capabilities element.

EID 255 indicates the WLAN_EID_EXT_* values.

EID 127 indicates the WLAN_EXT_CAPAB_* values.

So, I may have been confused with wpa_supplicant's use of _EXT_ to mean both "EXTension field" and "EXTended Capabilities".

I will need to research further to see if there is a third function for checking IEs with WLAN_EXT_CAPAB_* values.  Or if there should be.

That said, I can confirm that this patch is associating with both the OPEN secondary and the WPA primary (given appropriate config wpa_supplicant.conf).  Perhaps that was just a coincidence.

I need to go out shortly to run some errands for a few hours but will do some additional analysis and testing when I get back.
Comment 176 J.R. Oldroyd 2022-07-02 14:59:10 UTC
Additional info before I go...

If I use the original code and allow the ext_capab IE to be added in the "Workaround" section, these are the 13 octets that are added.  Values in (hex, dec):

DEBUG wpas_build_ext_capab: pos[0]=(0x7f, 127)    WLAN_EID_EXT_CAPAB
DEBUG wpas_build_ext_capab: pos[1]=(0x0b, 11)     size
DEBUG wpas_build_ext_capab: pos[2]=(0x00, 0)
DEBUG wpas_build_ext_capab: pos[3]=(0x00, 0)
DEBUG wpas_build_ext_capab: pos[4]=(0x0a, 10)
DEBUG wpas_build_ext_capab: pos[5]=(0x02, 2)
DEBUG wpas_build_ext_capab: pos[6]=(0x00, 0)
DEBUG wpas_build_ext_capab: pos[7]=(0x40, 64)
DEBUG wpas_build_ext_capab: pos[8]=(0x40, 64)
DEBUG wpas_build_ext_capab: pos[9]=(0x00, 0)
DEBUG wpas_build_ext_capab: pos[10]=(0x00, 0)
DEBUG wpas_build_ext_capab: pos[11]=(0x00, 0)
DEBUG wpas_build_ext_capab: pos[12]=(0x20, 32)

Somehow, something later is interpreting this as requiring to set WPA.
Comment 177 Cy Schubert freebsd_committer freebsd_triage 2022-07-02 15:35:07 UTC
I started from scratch, again.

Try this:

1. rm -rf /usr/obj/usr/src/amd64.amd64/usr.sbin/wpa
2. cd /usr/src/usr.sbin/wpa
3. make obj
4. make depend
5. make includes
6. make
7. make install

Run the test again.
Comment 178 Cy Schubert freebsd_committer freebsd_triage 2022-07-02 15:44:52 UTC
If you're running 13.1-RELEASE, add the following to usr.sbin/wpa/Makefile.inc

diff --git a/usr.sbin/wpa/Makefile.inc b/usr.sbin/wpa/Makefile.inc
index 5ce7dee4d2c5..24098bafb002 100644
--- a/usr.sbin/wpa/Makefile.inc
+++ b/usr.sbin/wpa/Makefile.inc
@@ -66,6 +66,7 @@ CFLAGS+=-DCONFIG_WNM
 CFLAGS+=-DCONFIG_WNM_AP
 CFLAGS+=-DCONFIG_MBO
 CFLAGS+=-DCONFIG_RSN_PREAUTH
+CFLAGS+=-O0
 
 .if ${MK_WPA_SUPPLICANT_EAPOL} != "no"
 CFLAGS+=-DCONFIG_HS20 \


After removing /usr/obj/usr/src/amd64.amd64/usr.sbin/wpa I can no longer reproduce this problem. I have just done the same to my everyday "prod" /usr/obj to verify the result.
Comment 179 Cy Schubert freebsd_committer freebsd_triage 2022-07-02 18:17:11 UTC
The above instructions only work for iwn(4). Unfortunately rtwn_usb(4) is still broken regardless.
Comment 180 Jaskie 2022-07-03 01:39:57 UTC
(In reply to Cy Schubert from comment #179)

Do you want my test result on my ath? Do I just have to change usr.sbin/wpa/Makefile.inc and no patches?
Comment 181 Cy Schubert freebsd_committer freebsd_triage 2022-07-03 03:23:08 UTC
(In reply to Jaskie from comment #180)

Don't worry, it's no longer needed. I will be pushing the workaround patch sometime tomorrow and will MFC it to stable/13 and stable/12 three days later. re@ will merge it into releng/13 sometime after that to allow binary updates.

There are a number of Linux distros that also have similar, but not the same, bugs reported. Unfortunately their users have not provided as much significant detail as you have, for which I am very grateful to you for.

Currently Red Hat has a bug open for failure to associate (WPA2) with Broadcom driver. Another with a Dlink AP. While here at FreeBSD I can reproduce the problem with rtwn_usb(4) with a secondary VAP on my Netgear however my iwn(4) works with that VAP. (FreeBSD uses the BSD driver while Linux uses the nl80211 driver.)

The wpa_supplicant port will also contain the patch and will be MFHed to 2022Q3 immediately. The wpa_supplicant29 port will obviously not. Neither will the wpa_supplicant-devel port get the patch in order to allow me and others to continue to work on this bug. (I typically use the -devel port, with its bleeding edge wpa_supplicant, anyway, eating my own dogfood.)

I am grateful for your assistance, providing significant outputs (which reading other orgs PRs outputs are relatively sketchy). Your help and patience has been appreciated.
Comment 182 J.R. Oldroyd 2022-07-03 07:21:27 UTC
I think that the workaround patch is the wrong approach.

If there are many folk clamoring for a fix, then okay, commit it.  But I don't get the sense that this is that urgent is it?

What I found in yesterday and Friday's debugging is that the code in the area I am looking at, in wpas_populate_assoc_ies(), appears to be generating our IE for outbound transmission.  It isn't parsing the AP's incoming IE.  The section that adds the IE that breaks things has the comment:

         * Workaround: Add Extended Capabilities element only if the AP
         * included this element in Beacon/Probe Response frames. Some older
         * APs seem to have interoperability issues if this element is
         * included, so while the standard may require us to include the
         * element in all cases, it is justifiable to skip it to avoid
         * interoperability issues.

As Cy pointed out, this code has been there for ages.  So, the question is, why are we now triggering the addition of this Extended Capabilities IE when we apparently didn't in earlier versions?

And, a second question is, given that we do add this ExtCapab IE with the cotents that I showed in comment #176, why does that cause our own driver to select WPA mode?

Given that the comment says that adding this IE can be skipped, I would suggest that a better temporary fix is to put "#if 0 ... #endif" around this section of the code in wpas_populate_assoc_ies().

I am just sitting back down again today and will look more at these now.
Comment 183 J.R. Oldroyd 2022-07-03 08:35:36 UTC
Latest findings...

The code in wpas_populate_assoc_ies() in the "Workaround" section calls wpas_build_ext_capab() to create this extended capabilities IE.  wpas_populate_assoc_ies() in turn calls wpas_ext_capab_byte() to actually put the data into the IE.

In the 2.9 version, wpas_ext_capab_byte() fills the IE with all 0 values and then truncates the IE to zero length.  Hence no extended capabilities IE is created.

In the 2.10 version, wpas_ext_capab_byte() is actually populating data (see comment #176) into the IE.
Comment 184 J.R. Oldroyd 2022-07-03 08:52:49 UTC
And the difference in wpas_ext_capab_byte() is due to the fact that in 2.10 various configuration options such as CONFIG_MBO and CONFIG_WNM and several others are enabled while they appear to be disabled in the 2.9 version (at least in the port security/wpa_supplicant29 version that I am using to compare).
Comment 185 J.R. Oldroyd 2022-07-03 10:52:56 UTC
The populated IE which, according to the comments, appears to be intended as a response to be transmitted, is being handed to wpa_drv_associate() where it is handed to wpa_driver_bsd_associate().

wpa_driver_bsd_associate() has the following code:

        if (params->wpa_ie_len &&
            set80211param(drv, IEEE80211_IOC_WPA,
                          params->wpa_ie[0] == WLAN_EID_RSN ? 2 : 1) < 0)
                return -1;

In other words, if there is an IE, don't bother looking to see what IE it is, just set WPA.

Looks like this code needs improving to only set WPA if the IE is a WPA IE.

Or, is it that the Extended Capabilities IE that we just generated as a response to the AP shouldn't be handed to our own driver?
Comment 186 Cy Schubert freebsd_committer freebsd_triage 2022-07-03 11:01:02 UTC
(In reply to J.R. Oldroyd from comment #183)

You're talking about upstream commit 2e06cef80a, which worked around misbehaving MBO/OCE APs that use RSN without PMF. A full revert of that upstream commit, three days ago, did not resolve the issue.

And, reversing, also three days ago, all the changes in wpas_ext_capab_byte() between hostap_2_9 hostap_2_10 altered the IE by five bytes but also resulted in an association failure of rtwn(4) to the secondary VAP.

Both paths were investigated. You are free to investigate both these hypotheses again yourself however.
Comment 187 Cy Schubert freebsd_committer freebsd_triage 2022-07-03 11:06:46 UTC
(In reply to J.R. Oldroyd from comment #185)

That code has been there since 2008.

^6fc6879bd (Jouni Malinen        2008-02-27 17:34:43 -0800 1265)        if (params->wpa_ie_len &&
^6fc6879bd (Jouni Malinen        2008-02-27 17:34:43 -0800 1266)            set80211param(drv, IEEE80211_IOC_WPA,
^6fc6879bd (Jouni Malinen        2008-02-27 17:34:43 -0800 1267)                          params->wpa_ie[0] == WLAN_EID_RSN ? 2 : 1) < 0)
^6fc6879bd (Jouni Malinen        2008-02-27 17:34:43 -0800 1268)                return -1;
Comment 188 J.R. Oldroyd 2022-07-03 12:28:01 UTC
Yes, the code has been there for ages.

However, in 2.10 additional CONFIG_XYZ options are enabled which result in the generation of an IE that wasn't being generated before.  That IE is (correctly or incorrectly) being handed to our driver which is then incorrectly setting WPA as a result of the IE.

Here is another proposed patch.

I have taken the approach that, if the driver receives an IE, whether or not it should have received it, it should process the IE properly.  At the moment the driver just says "hey, we have an IE, so lets enable WPA".  If the IE is the generated WLAN_EID_EXT_CAPAB IE, it is not the correct behavior to enable WPA.  So in this proposed patch, I am checking to see if the IE is WLAN_EID_RSN or at least not WLAN_EID_EXT_CAPAB and only setting WPA in the appropriate case.

diff --git a/contrib/wpa/src/drivers/driver_bsd.c b/contrib/wpa/src/drivers/driver_bsd.c
index c455bc93103..8ead569c2ff 100644
--- a/contrib/wpa/src/drivers/driver_bsd.c
+++ b/contrib/wpa/src/drivers/driver_bsd.c
@@ -1257,19 +1257,22 @@ wpa_driver_bsd_associate(void *priv, struct wpa_driver_associate_params *params)
        if (wpa_driver_bsd_set_auth_alg(drv, params->auth_alg) < 0)
                ret = -1;
        /* XXX error handling is wrong but unclear what to do... */
-       if (wpa_driver_bsd_set_wpa_ie(drv, params->wpa_ie, params->wpa_ie_len) < 0)
+       if (params->wpa_ie_len && params->wpa_ie[0] == WLAN_EID_RSN &&
+           wpa_driver_bsd_set_wpa_ie(drv, params->wpa_ie, params->wpa_ie_len) < 0)
                return -1;
 
        privacy = !(params->pairwise_suite == WPA_CIPHER_NONE &&
            params->group_suite == WPA_CIPHER_NONE &&
            params->key_mgmt_suite == WPA_KEY_MGMT_NONE &&
-           params->wpa_ie_len == 0);
+           (params->wpa_ie_len == 0 || params->wpa_ie[0] != WLAN_EID_RSN)
+           );
        wpa_printf(MSG_DEBUG, "%s: set PRIVACY %u", __func__, privacy);
 
        if (set80211param(drv, IEEE80211_IOC_PRIVACY, privacy) < 0)
                return -1;
 
        if (params->wpa_ie_len &&
+           params->wpa_ie[0] != WLAN_EID_EXT_CAPAB &&
            set80211param(drv, IEEE80211_IOC_WPA,
                          params->wpa_ie[0] == WLAN_EID_RSN ? 2 : 1) < 0)
                return -1;
Comment 189 Bjoern A. Zeeb freebsd_committer freebsd_triage 2022-07-03 13:28:33 UTC
(In reply to J.R. Oldroyd from comment #188)

I think what you really want to do, rather than excluding the one element, is to call get_ie() and search for the one you are actually interested in checking?
Comment 190 Bjoern A. Zeeb freebsd_committer freebsd_triage 2022-07-03 13:37:20 UTC
(In reply to Bjoern A. Zeeb from comment #189)

There's a potential follow-up problem then that we need to make sure that the way net80211 gets the packets out is also correct.

I am thinking of ieee80211_add_wpa / ieee80211_add_rsn / various add_ie().

I don't know if there is any magic involved offhand in the way IEs are passed down and/or if they need any re-sorting or splitting in general (not in this specific case).  Also not sure if it'll eventually (in the future) become a problem of possibly overflowing firmware space for IEs and hence want to re-sort to make sure (more) essential ones are always in the front and present.

But let's start by fixing this wpa user space bug first.
Comment 191 J.R. Oldroyd 2022-07-03 13:50:54 UTC
Created attachment 235049 [details]
patch to have wpa_driver_bsd_associate() check for WLAN_EID_RSN IE before setting WPA

That's a better idea, Björn.

I've redone the patch to do it that way.  Uploading as an attachment this time since it is a bit longer now.
Comment 192 J.R. Oldroyd 2022-07-03 14:02:02 UTC
Created attachment 235050 [details]
patch to have wpa_driver_bsd_associate() check for WLAN_EID_RSN IE before setting WPA

Slightly cleaner version of the same.
Comment 193 J.R. Oldroyd 2022-07-03 14:34:17 UTC
Created attachment 235051 [details]
patch to have wpa_driver_bsd_associate() check for WLAN_EID_RSN IE before setting WPA

And now a version that extracts the WLAN_EID_RSN and then actually uses it rather than still using the whole original IE.

The last part of this code:
    set80211param(drv, IEEE80211_IOC_WPA, wpa_ie[0] == WLAN_EID_RSN ? 2 : 1)
will now always set value 2.  Does that break any WPA1 cases?  If so, what WLAN_EID value is used for WPA1?
Comment 194 Cy Schubert freebsd_committer freebsd_triage 2022-07-03 14:36:56 UTC
Both new patches are incorrect.

The first last night, in comment #188 resolves the secondary VAP issue but regresses association to the primary VAP configured with WPA-PSK.

The patch in comment #192 fails to associate to anything.
Comment 195 J.R. Oldroyd 2022-07-03 14:48:38 UTC
(In reply to Cy Schubert from comment #194)

Please try the 2nd update of the attachment patch (comment #193).  It is working for me for both the primary with WPA2 and the secondary OPEN.

But, the patch in comment #188 also worked for both for me.

In my limited setup here, I do not have the ability to configure the AP with WPA1-only, so I cannot test that.  Neither can I configure this AP for WEP, so I can't test that, either.  Can any of you test these?
Comment 196 Cy Schubert freebsd_committer freebsd_triage 2022-07-03 14:50:23 UTC
(In reply to J.R. Oldroyd from comment #195)

This associates to the primary and secondary VAPs however only the primary can dhclient obtain an IP, not through the secondary VAP. This BTW is the same problem I had with my first workaround provided to Jaskie.
Comment 197 J.R. Oldroyd 2022-07-03 14:58:16 UTC
(In reply to Cy Schubert from comment #196)

I am posting this reply while associated with the secondary OPEN VAP here.  My previous comment was while associated with the primary WPA2 VAP.

I do note that, occasionally, dhclient does seem to take a while to get the IP although mostly it is pretty quick.  I have been assuming this to be due to the distance between the room I am in and the AP (about 35m) and occasional noise in the environment.  This happens for both the WPA2 and OPEN VAPs here.
Comment 198 Cy Schubert freebsd_committer freebsd_triage 2022-07-03 14:59:46 UTC
I think what is needed is to remove the IE, similar to what replace_ie() does except to just remove it.
Comment 199 J.R. Oldroyd 2022-07-03 15:20:31 UTC
I'm not sure what exactly you mean by that.

But, in answer to my own question in comment #193, it looks like I now also need to check for wpa_bss_get_vendor_ie WPA_IE_VENDOR_TYPE in order to know if WPA1 is available.

I shall work on adding that now.
Comment 200 J.R. Oldroyd 2022-07-03 15:57:08 UTC
Created attachment 235053 [details]
patch to have wpa_driver_bsd_associate() check for WLAN_EID_RSN IE before setting WPA

Version 4 of this patch now also checks for the vendor_ie that indicates WPA1 is available.

I duplicated the code.  If WPA2/RSN is available, I set that.  Else if WPA1 is available, I set that.  There may be a more optimal way of coding it, e.g., by making a new function that is called from both RSN and WPA sections.

I found that my one AP here does have a mixed WPA1+WPA2 mode, so I set that for the primary VAP.  I have tested with config in wpa_supplicant.conf for both RSN and separately for WPA and it associates both ways.
Comment 201 Cy Schubert freebsd_committer freebsd_triage 2022-07-03 16:07:08 UTC
(In reply to J.R. Oldroyd from comment #200)

Good work! Works perfectly.

Ideally this should be done outside of the driver, but, we're in a bit of a time crunch. This should move ahead. We can post this to the w1.fi ML. They will probably take it.
Comment 202 Cy Schubert freebsd_committer freebsd_triage 2022-07-03 16:17:48 UTC
Proposed commit log message:

Author:     J.R. Oldroyd <fbsd@opal.com>
AuthorDate: Sat Jul 2 11:15:31 2022 -0700
Commit:     Cy Schubert <cy@FreeBSD.org>
CommitDate: Sun Jul 3 09:15:15 2022 -0700

    wpa_supplicant: Resolve secondary VAP association issue
    
    When using rtwn(4) Atheros AR9285, and others, association will fail on
    a secondary open unprotected VAP when the primary VAP is configured for
    WPA. I was able to reproduce the problem using my rtwn_usb(4) but not
    my iwn(4).
    
    A number of broadly similar, but not the same, bugs have been discussed
    on Red Hat's bugzilla, suggesting driver issues.
    
    PR:             264238
    Reported by:    Jaskie <jiangjun12321@gmail.com>
                    "J.R. Oldroyd" <fbsd@opal.com>
    Submitted by:   "J.R. Oldroyd" <fbsd@opal.com>
    MFC after:      3 days
Comment 203 Bjoern A. Zeeb freebsd_committer freebsd_triage 2022-07-03 16:40:25 UTC
(In reply to Cy Schubert from comment #202)

I think this is highly suggestive and I am not sure other drivers would have similar problems due to the other nature of the 802.11 stack.

Also avoid "I".

Preferably describe the actual cause in the commit message as well, not the observed problem.  The real problem here is that we started adding more IEs and a 15 year old check is no longer holding up enabling WPA even if it shouldn't because there was yet another IE at the front of a list.   (needs better wording).
Comment 204 J.R. Oldroyd 2022-07-03 16:53:07 UTC
Created attachment 235055 [details]
patch to have wpa_driver_bsd_associate() check for WLAN_EID_RSN IE before setting WPA

Here's a 5th version.

This is the same functionality as the 4th version.  I just moved the duplicated code into a new function wpa_driver_bsd_set_rsn_wpa_ie() which I then call from each of the RSN and WPA sections.

I have tested with my one AP with a primary VAP of WPA2/RSN+WPA and a secondary VAP which is OPEN.  Here are the three connections to show it associates in each way:

        regdomain ETSI country NL authmode WPA2/802.11i privacy ON

        regdomain ETSI country NL authmode WPA privacy ON deftxkey UNDEF
 
        regdomain ETSI country NL authmode OPEN privacy OFF txpower 17

And, yes, it gets an IP and passes traffic in all three cases.

Not tested: WEP.  This AP doesn't support WEP.

It'd be great to get feedback from Jackie to know that this works there, too.

Jackie, if you are able to test this, I am testing with the stable/13 code with none of the other earlier patches.  You should be able to reset to your release/13.1.1 point and try with just this patch.
Comment 205 J.R. Oldroyd 2022-07-03 17:00:17 UTC
FWIW, I have been testing using the iwm(4) driver.
Comment 206 Cy Schubert freebsd_committer freebsd_triage 2022-07-03 17:25:03 UTC
I took a look at the Red Hat bugzilla to attempt to duplicate it here.

Primary VAP is WPA2 RSN AES, secondary VAP is OPEN: works with the secondary VAP.

Primary VAP is WPA2 RSN AES, secondary VAP is WPA TKIP: it did not associate to the secondary VAP intermittently.

It's probably ok but will need a little more testing to make absolutely sure.
Comment 207 Cy Schubert freebsd_committer freebsd_triage 2022-07-03 19:19:30 UTC
Tested with WEP.

Comparing this with an archlinux driver_nl80211 IE bugfix, applying the patch to driver_bsd is appropriate.
Comment 208 groenveld 2022-07-03 20:17:39 UTC
(In reply to J.R. Oldroyd from comment #200)

Patch applied against security/wpa_supplicant-devel
and worked with Marriott_Lobby:
# sysrc wpa_supplicant_program="/usr/local/sbin/wpa_supplicant"
# sysrc ifconfig_wlan0="WPA SYNCDHCP"
# ifconfig wlan0 list scan|grep Lobby
Marriott_Lobby                    28:b3:71:9c:58:29    1   54M  -65:-96   100 ES   WME HTCAP BSSLOAD
Marriott_Lobby                    f8:e7:1e:88:c9:c9   11   54M  -46:-96   100 ES   WME HTCAP BSSLOAD
Marriott_Lobby                    28:b3:71:9e:0b:59   11   54M  -55:-96   100 ES   WME HTCAP BSSLOAD
Marriott_Lobby                    28:b3:71:9c:58:2d  149   54M  -67:-96   100 E    WME HTCAP BSSLOAD VHTCAP VHTOPMODE VHTPWRENV

# ifconfig wlan0|grep -v ether
wlan0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        inet 172.16.1.144 netmask 0xfffff800 broadcast 172.16.7.255
        groups: wlan
        ssid Marriott_Lobby channel 11 (2462 MHz 11g) bssid f8:e7:1e:88:c9:c9
        regdomain FCC country US authmode OPEN privacy OFF txpower 30 bmiss 7
        scanvalid 60 protmode CTS wme roaming MANUAL
        parent interface: iwlwifi0
        media: IEEE 802.11 Wireless Ethernet OFDM/54Mbps mode 11g
        status: associated
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>

Thank you!
John
groenveld@acm.org
Comment 209 commit-hook freebsd_committer freebsd_triage 2022-07-03 21:37:00 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=775611ea11db0973fd8b7aef0f5eb527308efd05

commit 775611ea11db0973fd8b7aef0f5eb527308efd05
Author:     J.R. Oldroyd <fbsd@opal.com>
AuthorDate: 2022-07-02 18:15:31 +0000
Commit:     Cy Schubert <cy@FreeBSD.org>
CommitDate: 2022-07-03 21:19:38 +0000

    wpa_supplicant: Resolve secondary VAP association issue

    Association will fail on a secondary open unprotected VAP when the
    primary VAP is configured for WPA. Examples of secondary VAPs are,
    hotels, universities, and commodity routers' guest networks.

    A broadly similar bug was discussed on Red Hat's bugzilla affecting
    association to a D-Link DIR-842.

    This suggests that as IEs were added to the 802.11 protocol the old code
    was increasingly inadaquate to handle the additional IEs, not only a
    secondary VAP.

    PR:             264238
    Reported by:    Jaskie <jiangjun12321@gmail.com>
                    "J.R. Oldroyd" <fbsd@opal.com>
    Submitted by:   "J.R. Oldroyd" <fbsd@opal.com>
    MFC after:      3 days

 contrib/wpa/src/drivers/driver_bsd.c | 65 ++++++++++++++++++++++++++----------
 1 file changed, 48 insertions(+), 17 deletions(-)
Comment 210 commit-hook freebsd_committer freebsd_triage 2022-07-03 21:38:02 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/ports/commit/?id=b3916c7a8d2599e99fabdc1735b095ff5a9f9381

commit b3916c7a8d2599e99fabdc1735b095ff5a9f9381
Author:     J.R. Oldroyd <fbsd@opal.com>
AuthorDate: 2022-07-03 21:18:40 +0000
Commit:     Cy Schubert <cy@FreeBSD.org>
CommitDate: 2022-07-03 21:33:18 +0000

    wpa_supplicant* hostapd*: Resolve secondary VAP association issue

    Association will fail on a secondary open unprotected VAP when the
    primary VAP is configured for WPA. Examples of secondary VAPs are,
    hotels, universities, and commodity routers' guest networks.

    A broadly similar bug was discussed on Red Hat's bugzilla affecting
    association to a D-Link DIR-842.

    This suggests that as IEs were added to the 802.11 protocol the old code
    was increasingly inadaquate to handle the additional IEs, not only a
    secondary VAP.

    This duplcates src commit 775611ea11db here in ports.

    PR:             264238
    Reported by:    Jaskie <jiangjun12321@gmail.com>
                    "J.R. Oldroyd" <fbsd@opal.com>
    Submitted by:   "J.R. Oldroyd" <fbsd@opal.com>
    MFH:            2022Q3

 net/hostapd-devel/Makefile                         |   1 +
 .../files/patch-src_drivers_driver__bsd.c          | 109 ++++++++++++++++++---
 net/hostapd/Makefile                               |   2 +-
 net/hostapd/files/patch-src_drivers_driver__bsd.c  | 107 +++++++++++++++++---
 security/wpa_supplicant-devel/Makefile             |   1 +
 .../files/patch-src_drivers_driver__bsd.c          | 109 ++++++++++++++++++---
 security/wpa_supplicant/Makefile                   |   2 +-
 .../files/patch-src_drivers_driver__bsd.c          | 107 +++++++++++++++++---
 8 files changed, 390 insertions(+), 48 deletions(-)
Comment 211 Jaskie 2022-07-04 02:47:41 UTC
(In reply to J.R. Oldroyd from comment #204)

Sadly, it does not seem work for me. This is what I did:

# cd /usr/src
# git reset --hard release/13.1.0

# patch -C -p1 < ~adam/235055.patch && patch -C -p1 < ~/235055.patch                                                                                     
Hmm...  Looks like a unified diff to me...                                                                                                                                                 
The text leading up to this was:                                                                                                                                                           
--------------------------                                                                                                                                                                 
|diff --git a/contrib/wpa/src/drivers/driver_bsd.c b/contrib/wpa/src/drivers/driver_bsd.c                                                                                                  
|index c455bc93103..345bbb892ec 100644                                                                                                                                                     
|--- a/contrib/wpa/src/drivers/driver_bsd.c                                                                                                                                                
|+++ b/contrib/wpa/src/drivers/driver_bsd.c                                                                                                                                                
--------------------------                                                                                                                                                                 
Patching file contrib/wpa/src/drivers/driver_bsd.c using Plan A...                                                                                                                         
Hunk #1 succeeded at 14.                                                                                                                                                                   
Hunk #2 succeeded at 1201.                                                                                                                                                                 
Hunk #3 succeeded at 1282 with fuzz 2 (offset -3 lines).                                                                                                                                   
done                                                                                                                                                                                       
Hmm...  Looks like a unified diff to me...                                                                                                                                                 
The text leading up to this was:                                                                                                                                                           
--------------------------                                                                                                                                                                 
|diff --git a/contrib/wpa/src/drivers/driver_bsd.c b/contrib/wpa/src/drivers/driver_bsd.c                                                                                                  
|index c455bc93103..345bbb892ec 100644                                                                                                                                                     
|--- a/contrib/wpa/src/drivers/driver_bsd.c                                                                                                                                                
|+++ b/contrib/wpa/src/drivers/driver_bsd.c                                                                                                                                                
--------------------------                                                                                                                                                                 
Patching file contrib/wpa/src/drivers/driver_bsd.c using Plan A...                                                                                                                         
Hunk #1 succeeded at 14.                                                                                                                                                                   
Hunk #2 succeeded at 1201.                                                                                                                                                                 
Hunk #3 succeeded at 1282 with fuzz 2 (offset -3 lines).                                                                                                                                   
done                                          

# echo $?                                                      
0 

# make obj
# make depend
# make includes
# make
# make install


# service netif start
# ifconfig wlan0
wlan0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        ether 1c:4b:d6:ca:3e:ac
        inet 0.0.0.0/8 broadcast 255.255.255.255
        groups: wlan
        ssid WHU-STU channel 5 (2432 MHz 11g ht/20) bssid 70:d9:31:0e:2c:00
        regdomain 96 indoor ecm authmode WPA privacy MIXED deftxkey UNDEF
        txpower 20 bmiss 7 scanvalid 60 protmode CTS ampdulimit 64k shortgi
        -stbctx stbcrx -ldpc -uapsd wme burst roaming MANUAL bintval 300
        parent interface: ath0
        media: IEEE 802.11 Wireless Ethernet DS/1Mbps mode 11ng
        status: associated
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>

dhclient wlan0 didn't help either.
Comment 212 Cy Schubert freebsd_committer freebsd_triage 2022-07-04 03:25:20 UTC
(In reply to Jaskie from comment #211)

patch -C only checks the patch. And running patch -C twice checks it twice. It doesn't install the patch.

cd /usr/src
git apply /some/patch/file

The make obj and following instructions remain the same.
Comment 213 Cy Schubert freebsd_committer freebsd_triage 2022-07-04 03:33:36 UTC
(In reply to Cy Schubert from comment #212)
There are a number of missing steps. Let's document an approach to merge the patch from HEAD into your tree.

cd /usr/src
git fetch --all
git cherry-pick 775611ea11db
cd usr.sbin/wpa
make obj
make clean
make depend
make includes
make
make install
Comment 214 Jaskie 2022-07-04 03:57:34 UTC
(In reply to Cy Schubert from comment #213)

Ah sorry..

Confirmed, it WORKED! Thanks for your help!
Comment 215 commit-hook freebsd_committer freebsd_triage 2022-07-06 00:32:42 UTC
A commit in branch stable/13 references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=30d9a7b643dfa244319945b9f44c9d4f595b48ac

commit 30d9a7b643dfa244319945b9f44c9d4f595b48ac
Author:     J.R. Oldroyd <fbsd@opal.com>
AuthorDate: 2022-07-02 18:15:31 +0000
Commit:     Cy Schubert <cy@FreeBSD.org>
CommitDate: 2022-07-06 00:31:32 +0000

    wpa_supplicant: Resolve secondary VAP association issue

    Association will fail on a secondary open unprotected VAP when the
    primary VAP is configured for WPA. Examples of secondary VAPs are,
    hotels, universities, and commodity routers' guest networks.

    A broadly similar bug was discussed on Red Hat's bugzilla affecting
    association to a D-Link DIR-842.

    This suggests that as IEs were added to the 802.11 protocol the old code
    was increasingly inadaquate to handle the additional IEs, not only a
    secondary VAP.

    PR:             264238
    Reported by:    Jaskie <jiangjun12321@gmail.com>
                    "J.R. Oldroyd" <fbsd@opal.com>
    Submitted by:   "J.R. Oldroyd" <fbsd@opal.com>

    (cherry picked from commit 775611ea11db0973fd8b7aef0f5eb527308efd05)

 contrib/wpa/src/drivers/driver_bsd.c | 65 ++++++++++++++++++++++++++----------
 1 file changed, 48 insertions(+), 17 deletions(-)
Comment 216 commit-hook freebsd_committer freebsd_triage 2022-07-06 00:33:46 UTC
A commit in branch 2022Q3 references this bug:

URL: https://cgit.FreeBSD.org/ports/commit/?id=fcc343d18fd6aaeae96ae31aa7b1406bcea518ed

commit fcc343d18fd6aaeae96ae31aa7b1406bcea518ed
Author:     J.R. Oldroyd <fbsd@opal.com>
AuthorDate: 2022-07-03 21:18:40 +0000
Commit:     Cy Schubert <cy@FreeBSD.org>
CommitDate: 2022-07-06 00:31:53 +0000

    wpa_supplicant* hostapd*: Resolve secondary VAP association issue

    Association will fail on a secondary open unprotected VAP when the
    primary VAP is configured for WPA. Examples of secondary VAPs are,
    hotels, universities, and commodity routers' guest networks.

    A broadly similar bug was discussed on Red Hat's bugzilla affecting
    association to a D-Link DIR-842.

    This suggests that as IEs were added to the 802.11 protocol the old code
    was increasingly inadaquate to handle the additional IEs, not only a
    secondary VAP.

    This duplcates src commit 775611ea11db here in ports.

    PR:             264238
    Reported by:    Jaskie <jiangjun12321@gmail.com>
                    "J.R. Oldroyd" <fbsd@opal.com>
    Submitted by:   "J.R. Oldroyd" <fbsd@opal.com>

    (cherry picked from commit b3916c7a8d2599e99fabdc1735b095ff5a9f9381)

 net/hostapd-devel/Makefile                         |   1 +
 .../files/patch-src_drivers_driver__bsd.c          | 109 ++++++++++++++++++---
 net/hostapd/Makefile                               |   2 +-
 net/hostapd/files/patch-src_drivers_driver__bsd.c  | 107 +++++++++++++++++---
 security/wpa_supplicant-devel/Makefile             |   1 +
 .../files/patch-src_drivers_driver__bsd.c          | 109 ++++++++++++++++++---
 security/wpa_supplicant/Makefile                   |   2 +-
 .../files/patch-src_drivers_driver__bsd.c          | 107 +++++++++++++++++---
 8 files changed, 390 insertions(+), 48 deletions(-)
Comment 217 commit-hook freebsd_committer freebsd_triage 2022-07-06 00:33:47 UTC
A commit in branch stable/12 references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=7c38194341bf39e8878e3a88838aac7880825c02

commit 7c38194341bf39e8878e3a88838aac7880825c02
Author:     J.R. Oldroyd <fbsd@opal.com>
AuthorDate: 2022-07-02 18:15:31 +0000
Commit:     Cy Schubert <cy@FreeBSD.org>
CommitDate: 2022-07-06 00:32:53 +0000

    wpa_supplicant: Resolve secondary VAP association issue

    Association will fail on a secondary open unprotected VAP when the
    primary VAP is configured for WPA. Examples of secondary VAPs are,
    hotels, universities, and commodity routers' guest networks.

    A broadly similar bug was discussed on Red Hat's bugzilla affecting
    association to a D-Link DIR-842.

    This suggests that as IEs were added to the 802.11 protocol the old code
    was increasingly inadaquate to handle the additional IEs, not only a
    secondary VAP.

    PR:             264238
    Reported by:    Jaskie <jiangjun12321@gmail.com>
                    "J.R. Oldroyd" <fbsd@opal.com>
    Submitted by:   "J.R. Oldroyd" <fbsd@opal.com>

    (cherry picked from commit 775611ea11db0973fd8b7aef0f5eb527308efd05)

 contrib/wpa/src/drivers/driver_bsd.c | 65 ++++++++++++++++++++++++++----------
 1 file changed, 48 insertions(+), 17 deletions(-)