Created attachment 227932 [details] log file After updating stable/13 to the most recent version I am no more able to connect to EAP/PEAP secured WiFi network. Authentication goes fine but at the end wpa_supplicant dumps core leaving wlan0 with "no carrier" status. Connecting to WPA/WPA2 secured WiFi network with the same wpa_supplicant is still possible. To workaround this one has to install security/wpa_supplicant from ports which works flawlessly.
I am not able to attach the core file in bugzilla, so I am posting the link for downloading: https://pb.pwste.edu.pl/wpa_supplicant.core.gz
(In reply to Marek Zarychta from comment #0) Can you try wpa_supplicant-devel from ports. It's the same codebase and will tell us if it's the base software or the FreeBSD implementation.
(In reply to Cy Schubert from comment #2) My mistake, stable/13 is still at the same codebase as wpa-supplicant [without the -devel].
(In reply to Cy Schubert from comment #2) I have not tried the devel- branch but can check it as soon as will have the opportunity to do so
Can you give me uname -a please. I will try to make my stable/13 system the same as yours in order to process the dump. Otherwise you will need to be my eyes and hands.
(In reply to Cy Schubert from comment #5) #uname -a FreeBSD hplaptop.potoki.eu 13.0-STABLE FreeBSD 13.0-STABLE #10 stable/13-n247106-783ace22648: Mon Sep 6 11:02:44 CEST 2021 root@hamal.dom.potoki.eu:/usr/obj/usr/src/amd64.amd64/sys/MINTAKA amd64 At the moment I can't help with debugging this core. The world was installed without debugging files. I have updated to the most recent sources today and now I am rebuilding with debugging symbols enabled, so new cores will become debuggable in place.
(In reply to Marek Zarychta from comment #6) The problem is here: 375 if (sm->proto == WPA_PROTO_RSN && 376 !wpa_key_mgmt_suite_b(sm->key_mgmt) && 377 !wpa_key_mgmt_ft(sm->key_mgmt)) { But, all the variables have been optimized out by the compiler. The quick solution to this is: rm -rf /usr/obj/opt/src/git-src/amd64.amd64/usr.sbin/wpa cd /usr/src/usr.sbin/wpa DEBUG_FLAGS='-O0 -g' make make install Then try wpa_supplicant again and send me the dump. Don't update your system. I have built the same stable/13 with the same git hash as you have so I can examine your core dumps. sm or one of its members has not been initialized.
(In reply to Cy Schubert from comment #7) >Don't update your system. Too late. I am now on stable/13-n247303-adfb7f807c6. >Then try wpa_supplicant again and send me the dump. I have a few, will send you email and share them. Here's one of them: Reading symbols from /usr/sbin/wpa_supplicant... Reading symbols from /usr/lib/debug//usr/sbin/wpa_supplicant.debug... [New LWP 100330] Core was generated by `/usr/sbin/wpa_supplicant -s -dd -B -i wlan0 -c /etc/wpa_supplicant.conf -D bsd -'. Program terminated with signal SIGSEGV, Segmentation fault. #0 0x0000008000000076 in ?? () (gdb) bt #0 0x0000008000000076 in ?? () #1 0x0000000000316d93 in wpa_sm_key_mgmt_set_pmk (sm=0x801012700, pmk=0x801012700 "\354\350\206e\216\324\v\231ߞ\225\006\001\237\034N\355\362<h\034\204\342\222\330\367\354\023\272\305", <incomplete sequence \365>, pmk_len=32) at /usr/src/contrib/wpa/src/rsn_supp/wpa_i.h:393 #2 0x0000000000316cf4 in wpa_supplicant_key_mgmt_set_pmk (sm=0x801012700) at /usr/src/contrib/wpa/src/rsn_supp/wpa.c:252 #3 0x000000000031674a in wpa_supplicant_get_pmk (sm=0x801012700, src_addr=0x8010634a0 "", pmkid=0x8010444e9 "b\217\306[r\346\032\264\327\375\231\331Z!WC") at /usr/src/contrib/wpa/src/rsn_supp/wpa.c:374 #4 0x000000000031150a in wpa_supplicant_process_1_of_4 (sm=0x801012700, src_addr=0x8010634a0 "", key=0x801044484, ver=2, key_data=0x8010444e3 "\335\024", key_data_len=22) at /usr/src/contrib/wpa/src/rsn_supp/wpa.c:621 #5 0x000000000030ffca in wpa_sm_rx_eapol (sm=0x801012700, src_addr=0x8010634a0 "", buf=0x8010634a8 "\001\003", len=121) at /usr/src/contrib/wpa/src/rsn_supp/wpa.c:2438 #6 0x00000000002b91e2 in wpa_supplicant_rx_eapol (ctx=0x801039000, src_addr=0x8010634a0 "", buf=0x8010634a8 "\001\003", len=121) at /usr/src/contrib/wpa/wpa_supplicant/wpa_supplicant.c:4303 #7 0x0000000000308405 in l2_packet_receive (sock=6, eloop_ctx=0x801028be0, sock_ctx=0x801060000) at /usr/src/contrib/wpa/src/l2_packet/l2_packet_freebsd.c:98 #8 0x0000000000367460 in eloop_sock_table_dispatch (table=0x3702a0 <eloop+8>, fds=0x801044100) at /usr/src/contrib/wpa/src/utils/eloop.c:600 #9 0x00000000003670c2 in eloop_run () at /usr/src/contrib/wpa/src/utils/eloop.c:1223 #10 0x00000000002bc515 in wpa_supplicant_run (global=0x80102d000) at /usr/src/contrib/wpa/wpa_supplicant/wpa_supplicant.c:6526 #11 0x000000000029e5a1 in main (argc=12, argv=0x7fffffffebc0) at /usr/src/contrib/wpa/wpa_supplicant/main.c:397 quit)
(In reply to Marek Zarychta from comment #8) Did you build it with -O0? Do you have a dump? Can I have it? Looks like this time it failed at wpa_i.h:393 instead of wpa.c:375. I need more than a simple backtrace. What is the URL for the complete dump?
(In reply to Cy Schubert from comment #9) >Did you build it with -O0? I have built it with "DEBUG_FLAGS='-O0 -g' make" as suggested above, but with "CFLAGS=-O2 -pipe -fno-strict-aliasing" in make.conf(5) >What is the URL for the complete dump? I have sent URL to some dumps in e-mail message to cy@freebsd.org
(In reply to Cy Schubert from comment #9) Thanks. Spamassassin put it in my spam folder. I'll reply there.
Created attachment 227964 [details] Fix this PR Can you try this patch, please.
Created attachment 227990 [details] This should fix it. This patch should fix it.
(In reply to Cy Schubert from comment #13) Thank you for the patch. The latest one applies and builds fine. I will be able to test whether it solves the issue or not within few days and will give feedback here.
(In reply to Marek Zarychta from comment #14) Thank you. Keep me posted.
We should all take it easy, it is a minor breakage and probably no other setups except mine are affected. On the other hand, the security/wpa_supplicant from ports still works as intended. The QA process takes some time, the same for debugging and writing patches, but also taking dumps and testing patches requires some time and effort including either driving tens of miles or recreating EAP/PEAP secured network in the home lab. I have no insight into @freebsd.org e-mail server setup, but believe it does some false positives marking messages as SPAM what makes writing direct email messages a bit hopeless. The first conclusion: "no other setups are probably affected" is pretty sad and means that FreeBSD's user base became FreeBSD's consumer base. I have reported this breakage a week or two ago on the net@ mailing list and there was no feedback at all. The FreeBSD consumer base utilizes probably only RELEASEs and cares neither for the development process nor for the quality of the upcoming product. The final question which comes here is: Do we really need wpa_supplicant in the base? I was against ftpd(8) removal which IMHO is an imminent part of the FreeBSD OS, but wpa_supplicant can be easily installed from ports. Consumers who have only WiFi access can have the package on the USB stick. Kind regards, FreeBSD User
(In reply to Marek Zarychta from comment #16) > We should all take it easy, it is a minor breakage and probably no other setups except mine are affected. On the other hand, the security/wpa_supplicant from ports still works as intended. The problem is similar to the one fixed in -CURRENT during testing of the import of WPA that supported WiFi 6. It was fixed there during testing however this PR clearly shows that the problem wasn't the WiFi 6 wpa but instead the restructuring. The fix is the same. The fix will be MFCed into stable/12 and stable/13 in about two months anyway. It's the same fix that I posted here that I should have bisected and committed before committing WiFi 6 but I understood at the time that this was a WiFi 6 wpa problem but in fact it was with how I structured the Makefiles. The fix is an MFC of a tiny part of what will be MFCed. > The QA process takes some time, the same for debugging and writing patches, but also taking dumps and testing patches requires some time and effort including either driving tens of miles or recreating EAP/PEAP secured network in the home lab. This is appreciated. I had access to EAP/PEAP at $JOB but now we are #WFH. The office is permanently closed and I work from home now. The commute is great but access to EAP/PEAP not so great. > I have no insight into @freebsd.org e-mail server setup, but believe it does some false positives marking messages as SPAM what makes writing direct email messages a bit hopeless. FreeBSD mail servers don't block spam. I have a .forward at freebsd.org which forwards to my cschubert.com, which runs spamassassin on my mail gateway. It add SPAM headers which are read by procmail on my laptop which files emails into a SPAM folder. This is totally my fault. I should have checked my spam folder. I'm sorry. As to why spamassassin believes your emails are SPAM, Content analysis details: (5.5 points, 5.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- 0.2 BAYES_999 BODY: Bayes spam probability is 99.9 to 100% [score: 1.0000] 3.5 BAYES_99 BODY: Bayes spam probability is 99 to 100% [score: 1.0000] 0.0 SPF_NONE SPF: sender does not publish an SPF Record 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record 2.0 DEAR_SOMETHING BODY: Contains 'Dear (something)' -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.0 NICE_REPLY_A Looks like a legit reply (A) > The first conclusion: "no other setups are probably affected" is pretty sad and means that FreeBSD's user base became FreeBSD's consumer base. I have reported this breakage a week or two ago on the net@ mailing list and there was no feedback at all. The FreeBSD consumer base utilizes probably only RELEASEs and cares neither for the development process nor for the quality of the upcoming product. Sadly many people run -RELEASE. > The final question which comes here is: Do we really need wpa_supplicant in the base? I was against ftpd(8) removal which IMHO is an imminent part of the FreeBSD OS, but wpa_supplicant can be easily installed from ports. Consumers who have only WiFi access can have the package on the USB stick. Yes, IMO client utilities should be in base. Server daemons such as hostapd, krb5kdc, and the like probably should not be. Though, pkgbase would solve a lot of this. Developer maintenance time is another factor. Mozilla and Google have deprecated FTP from their browsers. Soon the only FTP clients will be the command line clients and those in utilities like filezilla. Lastly, at $JOB, our F5 does not support FTP. F5 has removed the FTP protocol from their Internet Gateway product line. I think that eventually FTP won't be supported anywhere. It's fine if we wait until the new WPA is MFCed. EAP/PEAP is probably not as used now that people (like me and others) work from home, permanently. I'll have to set up EAP/PEAP here. I can use either one of my computers downstairs as I use hostapd on it to test or I have a AP that can be configured to use radius. I'll need to set up a radius server with Kerberos authentication to test. Probably a good idea anyway.
I had an opportunity to test it today and can confirm that the patch solves the issue for me. It's only a minor breakage and not so many people are affected, so we can wait a bit for the MFC, but the decision of timings and further closing this bug I leave over to the committer. Anyway, thank you very much for unbreaking this in an expedited way.
Thank you for taking the time to test the patch. It is appreciated. I haven't decided yet whether to MFC part of c1d255d3ff (the part that is posted here) earlier than Nov 3.
Just so you know, I'm affected, too. At least it looks like exactly the same problem too me. Right now I'm using my phones Wifi via urndis to post this. # lldb /usr/sbin/wpa_supplicant (lldb) target create "/usr/sbin/wpa_supplicant" Current executable set to '/usr/sbin/wpa_supplicant' (x86_64). (lldb) run -i wlan0 -c /etc/wpa_supplicant.conf Process 2100 launched: '/usr/sbin/wpa_supplicant' (x86_64) Successfully initialized wpa_supplicant ioctl[SIOCS80211, op=20, val=0, arg_len=7]: Invalid argument ioctl[SIOCS80211, op=20, val=0, arg_len=7]: Invalid argument wlan0: Trying to associate with xx:xx:xx:xx:xx:xx (SSID='xxxxxxxx' freq=2412 MHz) Failed to add supported operating classes IE wlan0: Associated with xx:xx:xx:xx:xx:xx wlan0: CTRL-EVENT-EAP-STARTED EAP authentication started wlan0: CTRL-EVENT-EAP-PROPOSED-METHOD vendor=0 method=25 wlan0: CTRL-EVENT-EAP-METHOD EAP vendor 0 method 25 (PEAP) selected wlan0: CTRL-EVENT-EAP-PEER-CERT depth=1 subject='/C=XX/L=Xxxxxxxx/O=Xxxxxxx A/S/CN=Danfoss Intermediate' hash=3bd98e88f7577e8b90023e91a20b80af290b1713ed8ff07c95b792f516823a3f wlan0: CTRL-EVENT-EAP-PEER-CERT depth=0 subject='/CN=XXXXXXXXX.xxxxxxxxx.xxx' hash=4629a4c514ab0635d965018515d30253bc60071699067c0cb6af92e58b0a37e8 wlan0: CTRL-EVENT-EAP-PEER-ALT depth=0 DNS:XXXXXXXX.xxxxxxxx.xxx EAP-MSCHAPV2: Authentication succeeded EAP-TLV: TLV Result - Success - EAP-TLV/Phase2 Completed wlan0: CTRL-EVENT-EAP-SUCCESS EAP authentication completed successfully Process 2100 stopped * thread #1, name = 'wpa_supplicant', stop reason = signal SIGSEGV: invalid address (fault address: 0x8000000076) frame #0: 0x0000008000000076 error: memory read failed for 0x8000000000 (lldb) bt * thread #1, name = 'wpa_supplicant', stop reason = signal SIGSEGV: invalid address (fault address: 0x8000000076) * frame #0: 0x0000008000000076 frame #1: 0x00000000002c59f0 wpa_supplicant`wpa_sm_rx_eapol [inlined] wpa_sm_key_mgmt_set_pmk(sm=0x0000000800e12700, pmk="\"\xe0m\xb8\U00000002b%\xb3\xe5\xd8\xf5r\xfe+\U0000001d\xbd-\xb8Yq\xa5M\xe3\xe3\x82\U00000013\x9f\xd1&\U0000000eJ\xfc", pmk_len=32) at wpa_i.h:393:9 frame #2: 0x00000000002c59e1 wpa_supplicant`wpa_sm_rx_eapol [inlined] wpa_supplicant_key_mgmt_set_pmk(sm=0x0000000800e12700) at wpa.c:252 frame #3: 0x00000000002c5993 wpa_supplicant`wpa_sm_rx_eapol at wpa.c:374 frame #4: 0x00000000002c58aa wpa_supplicant`wpa_sm_rx_eapol [inlined] wpa_supplicant_process_1_of_4(sm=<unavailable>, src_addr=<unavailable>, key=0x0000000800e64a04, ver=<unavailable>, key_data=<unavailable>, key_data_len=<unavailable>) at wpa.c:621 frame #5: 0x00000000002c58aa wpa_supplicant`wpa_sm_rx_eapol(sm=<unavailable>, src_addr="\xb4]P\x9e8@\x88\x8e\U00000001\U00000003", buf="\U00000001\U00000003", len=<unavailable>) at wpa.c:2438 frame #6: 0x0000000000291592 wpa_supplicant`wpa_supplicant_rx_eapol(ctx=0x0000000800e2e000, src_addr="\xb4]P\x9e8@\x88\x8e\U00000001\U00000003", buf="\U00000001\U00000003", len=121) at wpa_supplicant.c:4303:3 frame #7: 0x00000000002bf799 wpa_supplicant`l2_packet_receive(sock=<unavailable>, eloop_ctx=0x0000000800e25be0, sock_ctx=<unavailable>) at l2_packet_freebsd.c:98:2 frame #8: 0x00000000002fa187 wpa_supplicant`eloop_run [inlined] eloop_sock_table_dispatch(table=<unavailable>, fds=0x0000000800e64700) at eloop.c:600:4 frame #9: 0x00000000002fa132 wpa_supplicant`eloop_run at eloop.c:1223 frame #10: 0x0000000000293254 wpa_supplicant`wpa_supplicant_run(global=0x0000000800e2a000) at wpa_supplicant.c:6526:2 frame #11: 0x0000000000281a54 wpa_supplicant`main(argc=<unavailable>, argv=<unavailable>) at main.c:397:14 frame #12: 0x000000000025e0f0 wpa_supplicant`_start(ap=<unavailable>, cleanup=<unavailable>) at crt1_c.c:75:7 (lldb)
(In reply to Dominic Fandrey from comment #20) Does the posted patch fix your problem?
A commit in branch stable/13 references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=0ab6ecd1dda7b6194b7aa09f70f9c6a7049708e4 commit 0ab6ecd1dda7b6194b7aa09f70f9c6a7049708e4 Author: Cy Schubert <cy@FreeBSD.org> AuthorDate: 2021-10-05 21:54:06 +0000 Commit: Cy Schubert <cy@FreeBSD.org> CommitDate: 2021-10-05 22:12:38 +0000 wpa: Fix EAP/PEAP MSCHAPv2 authentication SIGSEGV 25ecdc7d52770caf1c9b44b5ec11f468f6b636f3 (MFCed by 13f32ff71eeb7213bb9f34bdfa88c7ccecf451bc) introduced a link error causing a SIGSEGV when using EAP/PEAP MSCHAPv2 authentication. It was subsequently addressed by c1d255d3ffdbe447de3ab875bf4e7d7accc5bfc5, discovered by build time link errors not experienced during testing of 25ecdc7d52770caf1c9b44b5ec11f468f6b636f3. This commit MFCs a portion of c1d255d3ffdbe447de3ab875bf4e7d7accc5bfc5 addressing only the SIGSEGV. The rest of c1d255d3ffdbe447de3ab875bf4e7d7accc5bfc5 will be MFCed in November 2021. This is a direct commit to stable/13. PR: 258527 Reported by: Marek Zarychta <zarychtam@plan-b.pwste.edu.pl> Tested by: Marek Zarychta <zarychtam@plan-b.pwste.edu.pl> usr.sbin/wpa/Makefile.crypto | 1 + usr.sbin/wpa/Makefile.inc | 5 +++-- usr.sbin/wpa/hostapd/Makefile | 2 -- usr.sbin/wpa/src/ap/Makefile | 9 +-------- usr.sbin/wpa/src/common/Makefile | 1 + usr.sbin/wpa/src/rsn_supp/Makefile | 4 ---- 6 files changed, 6 insertions(+), 16 deletions(-)
A commit in branch stable/12 references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=ad0311b6956f66835c24bd5a7e735aaa6b8c8852 commit ad0311b6956f66835c24bd5a7e735aaa6b8c8852 Author: Cy Schubert <cy@FreeBSD.org> AuthorDate: 2021-10-05 22:09:06 +0000 Commit: Cy Schubert <cy@FreeBSD.org> CommitDate: 2021-10-05 22:09:06 +0000 wpa: Fix EAP/PEAP MSCHAPv2 authentication SIGSEGV 25ecdc7d52770caf1c9b44b5ec11f468f6b636f3 (MFCed by 5654815fd3604e024eefdcb34904d3a7c883e8c5) introduced a link error causing a SIGSEGV when using EAP/PEAP MSCHAPv2 authentication. It was subsequently addressed by c1d255d3ffdbe447de3ab875bf4e7d7accc5bfc5, discovered by build time link errors not experienced during testing of 25ecdc7d52770caf1c9b44b5ec11f468f6b636f3. This commit MFCs a portion of c1d255d3ffdbe447de3ab875bf4e7d7accc5bfc5 addressing only the SIGSEGV. The rest of c1d255d3ffdbe447de3ab875bf4e7d7accc5bfc5 will be MFCed in November 2021. This is a direct commit to stable/12. PR: 258527 Reported by: Marek Zarychta <zarychtam@plan-b.pwste.edu.pl> Tested by: Marek Zarychta <zarychtam@plan-b.pwste.edu.pl> usr.sbin/wpa/Makefile.crypto | 1 + usr.sbin/wpa/Makefile.inc | 5 +++-- usr.sbin/wpa/hostapd/Makefile | 2 -- usr.sbin/wpa/src/ap/Makefile | 9 +-------- usr.sbin/wpa/src/common/Makefile | 1 + usr.sbin/wpa/src/rsn_supp/Makefile | 4 ---- 6 files changed, 6 insertions(+), 16 deletions(-)
A portion of c1d255d3ffdbe447de3ab875bf4e7d7accc5bfc5 addressing only the SIGSEGV has been MFCed. The rest of c1d255d3ffdbe447de3ab875bf4e7d7accc5bfc5 will be MFCed in November.
Thank you again for fixing the issue in stable/13.