Bug 233949

Summary: rtwn(4): RTL8192CU packet loss 10-25% (EDIMAX N150 Nano USB Adapter)
Product: Base System Reporter: mr_beaner_2003
Component: wirelessAssignee: Andriy Voskoboinyk <avos>
Status: Closed FIXED    
Severity: Affects Many People CC: avos, frimen.c, khanzf, mizhka, wireless
Priority: --- Keywords: performance
Version: 12.0-RELEASEFlags: koobs: mfc-stable12+
koobs: mfc-stable11+
Hardware: amd64   
OS: Any   
See Also: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=218527
Attachments:
Description Flags
Performance and changes none

Description mr_beaner_2003 2018-12-12 00:20:22 UTC
RE: 218527, is not fixed. I am testing with an EDIMAX N150 Nano USB Adapter.

# freebsd-version
13.0-CURRENT

# usbconfig -d ugen1.2 dump_all_desc 
ugen1.2: <Realtek 802.11n WLAN Adapter> at usbus1, cfg=0 md=HOST spd=HIGH (480Mbps) pwr=ON (500mA)

  bLength = 0x0012 
  bDescriptorType = 0x0001 
  bcdUSB = 0x0200 
  bDeviceClass = 0x0000  <Probed by interface class>
  bDeviceSubClass = 0x0000 
  bDeviceProtocol = 0x0000 
  bMaxPacketSize0 = 0x0040 
  idVendor = 0x7392 
  idProduct = 0x7811 
  bcdDevice = 0x0200 
  iManufacturer = 0x0001  <Realtek>
  iProduct = 0x0002  <802.11n WLAN Adapter>
  iSerialNumber = 0x0003  <00e04c000001>
  bNumConfigurations = 0x0001 

 Configuration index 0

    bLength = 0x0009 
    bDescriptorType = 0x0002 
    wTotalLength = 0x002e 
    bNumInterfaces = 0x0001 
    bConfigurationValue = 0x0001 
    iConfiguration = 0x0000  <no string>
    bmAttributes = 0x00a0 
    bMaxPower = 0x00fa 

    Interface 0
      bLength = 0x0009 
      bDescriptorType = 0x0004 
      bInterfaceNumber = 0x0000 
      bAlternateSetting = 0x0000 
      bNumEndpoints = 0x0004 
      bInterfaceClass = 0x00ff  <Vendor specific>
      bInterfaceSubClass = 0x00ff 
      bInterfaceProtocol = 0x00ff 
      iInterface = 0x0000  <no string>

     Endpoint 0
        bLength = 0x0007 
        bDescriptorType = 0x0005 
        bEndpointAddress = 0x0081  <IN>
        bmAttributes = 0x0002  <BULK>
        wMaxPacketSize = 0x0200 
        bInterval = 0x0000 
        bRefresh = 0x0000 
        bSynchAddress = 0x0000 

     Endpoint 1
        bLength = 0x0007 
        bDescriptorType = 0x0005 
        bEndpointAddress = 0x0002  <OUT>
        bmAttributes = 0x0002  <BULK>
        wMaxPacketSize = 0x0200 
        bInterval = 0x0000 
        bRefresh = 0x0000 
        bSynchAddress = 0x0000 

     Endpoint 2
        bLength = 0x0007 
        bDescriptorType = 0x0005 
        bEndpointAddress = 0x0003  <OUT>
        bmAttributes = 0x0002  <BULK>
        wMaxPacketSize = 0x0200 
        bInterval = 0x0000 
        bRefresh = 0x0000 
        bSynchAddress = 0x0000 

     Endpoint 3
        bLength = 0x0007 
        bDescriptorType = 0x0005 
        bEndpointAddress = 0x0084  <IN>
        bmAttributes = 0x0003  <INTERRUPT>
        wMaxPacketSize = 0x0040 
        bInterval = 0x0001 
        bRefresh = 0x0000 
        bSynchAddress = 0x0000 

# ifconfig wlan0
wlan0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
	ether 74:da:38:d5:f5:4b
	inet 172.30.8.76 netmask 0xffffffc0 broadcast 172.30.8.127 
	groups: wlan 
	ssid WAPNAME channel 11 (2462 MHz 11g ht/20) bssid 80:2a:a8:d7:ec:a5
	regdomain FCC country US authmode WPA2/802.11i privacy ON
	deftxkey UNDEF AES-CCM 2:128-bit txpower 30 bmiss 7 scanvalid 60
	protmode CTS ht20 ampdulimit 64k ampdudensity 8 shortgi -stbc -ldpc
	wme roaming MANUAL
	media: IEEE 802.11 Wireless Ethernet MCS mode 11ng
	status: associated
	nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>

# ping -c 100 172.30.8.65
PING 172.30.8.65 (172.30.8.65): 56 data bytes
64 bytes from 172.30.8.65: icmp_seq=0 ttl=255 time=16.389 ms
...
64 bytes from 172.30.8.65: icmp_seq=59 ttl=255 time=1316.844 ms
64 bytes from 172.30.8.65: icmp_seq=61 ttl=255 time=121.498 ms
64 bytes from 172.30.8.65: icmp_seq=62 ttl=255 time=44.221 ms
64 bytes from 172.30.8.65: icmp_seq=63 ttl=255 time=59.534 ms
64 bytes from 172.30.8.65: icmp_seq=64 ttl=255 time=66.640 ms
64 bytes from 172.30.8.65: icmp_seq=65 ttl=255 time=39.747 ms
64 bytes from 172.30.8.65: icmp_seq=66 ttl=255 time=57.837 ms
64 bytes from 172.30.8.65: icmp_seq=67 ttl=255 time=104.932 ms
...

--- 172.30.8.65 ping statistics ---
100 packets transmitted, 75 packets received, 25.0% packet loss
round-trip min/avg/max/stddev = 12.461/58.785/1316.844/147.781 ms
Comment 1 mr_beaner_2003 2018-12-12 00:24:31 UTC
# cat /boot/loader.conf 
kern.geom.label.disk_ident.enable="0"
kern.geom.label.gptid.enable="0"
zfs_load="YES"
legal.realtek.license_ack=1
if_rtwn_pci_load="YES"
if_rtwn_usb_load="YES"
pf_load="YES"

# cat /etc/rc.conf
...
ifconfig_em0="DHCP"
wlans_rtwn0="wlan0"
ifconfig_wlan0="WPA DHCP"
...
Comment 2 Andriy Voskoboinyk freebsd_committer freebsd_triage 2018-12-12 23:38:48 UTC
Hi,

Ping times are a bit unstable - how many APs in range? (ifconfig wlan0 list scan)

You can try to disable some (or all) 11n features - there may be a reason of packet loss:
1) ifconfig wlan0 -shortgi
2) ifconfig wlan0 -ampdutx
3) ifconfig wlan0 -amsdutx

If nothing helps, try to force 11g mode:
ifconfig wlan0 channel 11:g
Comment 3 mr_beaner_2003 2018-12-13 02:58:11 UTC
I have two SSIDs from one WAP,
# ifconfig wlan0 list scan
SSID/MESH ID                      BSSID              CHAN RATE    S:N     INT CAPS
WAPGUEST                       82:2a:a8:d7:ec:a5   11   54M  -81:-95   100 EPS  BSSLOAD HTCAP WME ATH RSN
WAPNAME                      80:2a:a8:d7:ec:a5   11   54M  -83:-95   100 EPS  BSSLOAD HTCAP WME ATH RSN
# ping -c 100 -q 172.30.8.65
PING 172.30.8.65 (172.30.8.65): 56 data bytes

--- 172.30.8.65 ping statistics ---
100 packets transmitted, 98 packets received, 2.0% packet loss
round-trip min/avg/max/stddev = 8.787/20.475/77.679/12.369 ms
#  
# ifconfig wlan0 -shortgi
# ping -c 100 -q 172.30.8.65
PING 172.30.8.65 (172.30.8.65): 56 data bytes

--- 172.30.8.65 ping statistics ---
100 packets transmitted, 99 packets received, 1.0% packet loss
round-trip min/avg/max/stddev = 9.907/31.190/1020.956/101.438 ms
# ifconfig wlan0
wlan0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
	ether 74:da:38:d5:f5:4b
	inet 172.30.8.76 netmask 0xffffffc0 broadcast 172.30.8.127 
	groups: wlan 
	ssid WAPNAME channel 11 (2462 MHz 11g ht/20) bssid 80:2a:a8:d7:ec:a5
	regdomain FCC country US authmode WPA2/802.11i privacy ON
	deftxkey UNDEF AES-CCM 2:128-bit txpower 30 bmiss 7 scanvalid 60
	protmode CTS ht20 ampdulimit 64k ampdudensity 8 -stbc -ldpc wme
	roaming MANUAL
	media: IEEE 802.11 Wireless Ethernet MCS mode 11ng
	status: associated
	nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
# ifconfig wlan0 -ampdutx
# ping -c 100 -q 172.30.8.65
PING 172.30.8.65 (172.30.8.65): 56 data bytes

--- 172.30.8.65 ping statistics ---
100 packets transmitted, 100 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 10.537/21.692/70.821/10.093 ms
# ifconfig wlan0 ampdutx
# ifconfig wlan0 amsdutx
# ping -c 100 -q 172.30.8.65
PING 172.30.8.65 (172.30.8.65): 56 data bytes

--- 172.30.8.65 ping statistics ---
100 packets transmitted, 99 packets received, 1.0% packet loss
round-trip min/avg/max/stddev = 11.616/31.129/1034.261/101.653 ms
# ping -c 100 -q 172.30.8.65
PING 172.30.8.65 (172.30.8.65): 56 data bytes
f
--- 172.30.8.65 ping statistics ---
100 packets transmitted, 99 packets received, 1.0% packet loss
round-trip min/avg/max/stddev = 11.069/29.829/1092.987/107.861 ms
# ifconfig wlan0 -amsdutx
# ping -c 100 -q 172.30.8.65
PING 172.30.8.65 (172.30.8.65): 56 data bytes

--- 172.30.8.65 ping statistics ---
100 packets transmitted, 99 packets received, 1.0% packet loss
round-trip min/avg/max/stddev = 10.609/28.940/1082.781/106.818 ms
# ifconfig wlan0 amsdutx
# ifconfig wlan0 ampdutx
# ifconfig wlan0 
wlan0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
	ether 74:da:38:d5:f5:4b
	inet 172.30.8.76 netmask 0xffffffc0 broadcast 172.30.8.127 
	groups: wlan 
	ssid WAPNAME channel 11 (2462 MHz 11g ht/20) bssid 80:2a:a8:d7:ec:a5
	regdomain FCC country US authmode WPA2/802.11i privacy ON
	deftxkey UNDEF AES-CCM 2:128-bit txpower 30 bmiss 7 scanvalid 60
	protmode CTS ht20 ampdulimit 64k ampdudensity 8 -stbc -ldpc wme
	roaming MANUAL
	media: IEEE 802.11 Wireless Ethernet MCS mode 11ng
	status: associated
	nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
# ifconfig wlan0 channel 11:g
# ifconfig wlan0
wlan0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
	ether 74:da:38:d5:f5:4b
	inet 172.30.8.76 netmask 0xffffffc0 broadcast 172.30.8.127 
	groups: wlan 
	ssid WAPNAME channel 11 (2462 MHz 11g ht/20) bssid 80:2a:a8:d7:ec:a5
	regdomain FCC country US authmode WPA2/802.11i privacy ON
	deftxkey UNDEF AES-CCM 2:128-bit txpower 30 bmiss 7 scanvalid 60
	protmode CTS ht20 ampdulimit 64k ampdudensity 8 -stbc -ldpc wme
	roaming MANUAL
	media: IEEE 802.11 Wireless Ethernet autoselect mode 11g
	status: associated
	nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
# ping -c 100 -q 172.30.8.65
PING 172.30.8.65 (172.30.8.65): 56 data bytes

--- 172.30.8.65 ping statistics ---
100 packets transmitted, 84 packets received, 16.0% packet loss
round-trip min/avg/max/stddev = 10.599/175.849/2100.622/391.322 ms
# 

Using the -ht flag doesn't improve performance, and I can replicate this issue across three different, identical NICs.
Comment 4 mr_beaner_2003 2018-12-27 23:11:21 UTC
I've run a PCAP on the gateway, as well as the host, and it would appear that it largely affects the receive queue and it's asymmetrical sends-receives. The packet loss is on the reception side.
Comment 5 mr_beaner_2003 2019-01-04 17:04:15 UTC
I think the tx/rx queues are jacked. I get 100% packet transfer in DragonFlyBSD and 100% in OpenBSD. I am unsure about the code re-architecture.
Comment 6 Farhan Khan 2019-01-09 06:14:33 UTC
Is there something you are doing to produce this issue? I do not have this problem on my end. I have an RTL8188EU and it works without packet loss for me.
Comment 7 mr_beaner_2003 2019-01-09 17:44:07 UTC
I am just performing an ICMP-echo to my gateway with the rtwn(4) driver. I have determined that it's neither the 80211 stack nor the NIC because I get 0% packet loss with DFlyBSD/OpenBSD and their urtwn(4) driver.

While performing the ICMP-echo with the gateway, I've captured a PCAP on the gateway and the FreeBSD host; I see FreeBSD sending the ICMP-echo, and I see the gateway receiving it and sending the ICMP-reply back. But on the FreeBSD/12-RELEASE and FreeBSD/13-CURRENT host, the ICMP-replies are not showing in the PCAP.

I am using an EDIMAX N150 (RTL8192CU) wireless NIC, as outlined in my usbconfig dump.
Comment 8 Andriy Voskoboinyk freebsd_committer freebsd_triage 2019-01-19 16:05:22 UTC
This is RTL8192CU-specific - I can reproduce the packet loss with RTL8188CUS, but not with others.
Comment 9 mr_beaner_2003 2019-02-01 05:03:15 UTC
Created attachment 201576 [details]
Performance and changes

I am unsure as to what is happening, but I referred to the DrangonflyBSD urtwn(4) driver for some guidance. It's odd that this change improved performance from 25% packet loss to ~1% packaet loss. Any ideas?
Comment 10 mr_beaner_2003 2019-02-04 21:07:00 UTC
It seems there are some data type mismatches, which may be resulting in undefined behavior. This is probably more appropriate, after reviewing the data structures...
Index: /usr/src/sys/dev/rtwn/rtl8192c/r92c_rx.c
===================================================================
--- /usr/src/sys/dev/rtwn/rtl8192c/r92c_rx.c    (revision 343552)
+++ /usr/src/sys/dev/rtwn/rtl8192c/r92c_rx.c    (working copy)
@@ -83,10 +83,10 @@
 r92c_get_rssi_ofdm(struct rtwn_softc *sc, void *physt)
 {
        struct r92c_rx_phystat *phy = (struct r92c_rx_phystat *)physt;
-       int rssi;
+       uint8_t rssi = phy->pwdb_all;
 
        /* Get average RSSI. */
-       rssi = ((phy->pwdb_all >> 1) & 0x7f) - 110;
+       rssi = ((rssi >> 1) & 0x7f) - 110;
 
        return (rssi);
 }
Comment 11 mr_beaner_2003 2019-02-08 13:47:20 UTC
If my diff @2019-02-04 21:07:00 UTC, proves to resolve packet loss on RTL8192CU and RTL8188CU, then this may be resolved. My RTL8192CU has an acceptable number of packet loss <1%.

I will however open another ticket regarding performance, as it's getting "dial-up speeds."
Comment 12 mr_beaner_2003 2019-02-21 16:31:53 UTC
Okay - this is confusing; Linux is reporting this device as RTL8188CUS, which is it?

Bus 006 Device 003: ID 7392:7811 Edimax Technology Co., Ltd EW-7811Un 802.11n Wireless Adapter [Realtek RTL8188CUS]
Comment 13 commit-hook freebsd_committer freebsd_triage 2019-03-04 03:04:22 UTC
A commit references this bug:

Author: avos
Date: Mon Mar  4 03:02:15 UTC 2019
New revision: 344745
URL: https://svnweb.freebsd.org/changeset/base/344745

Log:
  rtwn_usb(4): fix Tx instability with RTL8192CU chipsets

  - Fix data frames transmission via POWER_STATUS register setup -
  it seems to be set by MACID_CONFIG firmware command, which was broken*
  in r290439 and later disabled in r307529.

  We can re-enable it later if / when firmware rate adaptation will be
  ready; however, this step will be required anyway - for firmware-less
  builds.

  - Force RTS / CTS protection frame rate to CCK1 (this rate works fine
  without any additional setup; no better workaround is known yet).

  The problem was not observed on the channel 1 or with CCK1 rate enforced
  ('ifconfig wlan0 ucastrate 1' for 11 b/g; not possible for 11n networks
  due to ifconfig(8) bug).

  * I'm not sure if it works before r290439 because - AFAIR - I never seen
  firmware rate adaptation working for 10-STABLE urtwn(4)
  (It needs EN_BCN bit set and RSSI updates at least).

  Tested with RTL8188CUS in STA mode
  (in regular mode and with disabled MRR - DARFRC*8 is set to 0)

  PR:		233949
  MFC after:	2 weeks

Changes:
  head/sys/dev/rtwn/rtl8192c/r92c_reg.h
  head/sys/dev/rtwn/rtl8192c/r92c_tx.c
  head/sys/dev/rtwn/rtl8192c/usb/r92cu_init.c
Comment 14 Andriy Voskoboinyk freebsd_committer freebsd_triage 2019-03-06 08:38:58 UTC
Can you reproduce the issue with committed fix?
Comment 15 commit-hook freebsd_committer freebsd_triage 2019-03-18 02:57:26 UTC
A commit references this bug:

Author: avos
Date: Mon Mar 18 02:56:52 UTC 2019
New revision: 345253
URL: https://svnweb.freebsd.org/changeset/base/345253

Log:
  MFC r344745:
  rtwn_usb(4): fix Tx instability with RTL8192CU chipsets

  PR:		233949

Changes:
_U  stable/12/
  stable/12/sys/dev/rtwn/rtl8192c/r92c_reg.h
  stable/12/sys/dev/rtwn/rtl8192c/r92c_tx.c
  stable/12/sys/dev/rtwn/rtl8192c/usb/r92cu_init.c
Comment 16 commit-hook freebsd_committer freebsd_triage 2019-03-18 02:59:30 UTC
A commit references this bug:

Author: avos
Date: Mon Mar 18 02:58:35 UTC 2019
New revision: 345254
URL: https://svnweb.freebsd.org/changeset/base/345254

Log:
  MFC r344745:
  urtwn(4): fix Tx instability with RTL8192CU chipsets

  PR:		233949

Changes:
_U  stable/11/
  stable/11/sys/dev/urtwn/if_urtwn.c
  stable/11/sys/dev/urtwn/if_urtwnreg.h
Comment 17 mr_beaner_2003 2020-01-08 18:59:36 UTC
# uname -U; uname -K
1300074
1300074

# tail /var/log/messages
Jan  8 12:30:15 fb13c01 kernel: rtwn0: at uhub0, port 1, addr 1 (disconnected)
Jan  8 12:30:15 fb13c01 kernel: rtwn0: r92c_handle_c2h_task: C2H report 15 (len 15) was not handled
Jan  8 12:30:15 fb13c01 kernel: wlan0: link state changed to DOWN
Jan  8 12:30:15 fb13c01 kernel: rtwn0: detached
Jan  8 12:30:22 fb13c01 kernel: usbd_setup_device_desc: getting device descriptor at addr 1 failed, USB_ERR_TIMEOUT
Jan  8 12:30:23 fb13c01 kernel: usbd_req_re_enumerate: addr=1, set address failed! (USB_ERR_IOERROR, ignored)
Jan  8 12:30:29 fb13c01 kernel: usbd_setup_device_desc: getting device descriptor at addr 1 failed, USB_ERR_TIMEOUT
Jan  8 12:30:30 fb13c01 kernel: usbd_req_re_enumerate: addr=1, set address failed! (USB_ERR_IOERROR, ignored)
Jan  8 12:30:36 fb13c01 kernel: usbd_setup_device_desc: getting device descriptor at addr 1 failed, USB_ERR_TIMEOUT
Jan  8 12:30:36 fb13c01 kernel: usbd_req_re_enumerate: addr=1, set address failed! (USB_ERR_IOERROR, ignored)
Jan  8 12:30:42 fb13c01 kernel: usbd_setup_device_desc: getting device descriptor at addr 1 failed, USB_ERR_TIMEOUT
Jan  8 12:30:43 fb13c01 kernel: usbd_req_re_enumerate: addr=1, set address failed! (USB_ERR_IOERROR, ignored)
Jan  8 12:30:49 fb13c01 kernel: usbd_setup_device_desc: getting device descriptor at addr 1 failed, USB_ERR_TIMEOUT
Jan  8 12:30:49 fb13c01 kernel: ugen0.2: <Unknown > at usbus0 (disconnected)
Jan  8 12:30:49 fb13c01 kernel: uhub_reattach_port: could not allocate new device
Comment 18 Kubilay Kocak freebsd_committer freebsd_triage 2020-07-31 03:26:21 UTC
^Triage: 

- Assign to committer that resolved
- Track merges

@Reporter If this is still an issue, please re-open with additional details and steps to reproduce