Bug 274995

Summary: net/realtek-re-kmod: ssl3_get_record:decryption failed or bad record mac
Product: Ports & Packages Reporter: iio7
Component: Individual Port(s)Assignee: Alex Dupre <ale>
Status: Closed FIXED    
Severity: Affects Some People Flags: bugzilla: maintainer-feedback? (ale)
Priority: ---    
Version: Latest   
Hardware: amd64   
OS: Any   

Description iio7 2023-11-09 18:00:05 UTC
Hi,

I have this strange error on 2 out of 4 FreeBSD boxes which I have narrowed down to having relation to the realtek-re-kmod driver.

$ curl -O https://www.unixsheikh.com/includes/files/the-biggest-myths.pdf
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
12  763k   12 98139    0     0   377k      0  0:00:02 --:--:--  0:00:02  378k
curl: (56) OpenSSL SSL_read: OpenSSL/1.1.1t: error:1408F119:SSL routines:ssl3_get_record:decryption failed or bad record mac, errno 0

I have problems with IMAP, SFTP, and HTTPS using my email client, wget, FileZilla etc.

When I attach an Intel NIC with the igb driver on the same machine, the problem is gone.

Both boxes have Realtek NICs on the motherboard, none of which work with the re driver from base.

The error is strangely enough not on all connections to all HTTPS domains.

First broken box:

$ ifconfig re0
re0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=201b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,WOL_MAGIC>
        ether d0:50:99:59:15:7e
        inet 192.168.1.3 netmask 0xffffff00 broadcast 192.168.1.255
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>

# dmesg |grep re0
re0: <Realtek PCIe GbE Family Controller> port 0xe000-0xe0ff mem 0xd0704000-0xd0704fff,0xd0700000-0xd0703fff irq 18 at device 0.0 on pci1
re0: Using Memory Mapping!
re0: Using 1 MSI-X message
re0: ASPM disabled
re0: version:1.98.00
re0: Ethernet address: d0:50:99:59:15:7e
re0: Ethernet address: d0:50:99:59:15:7e
re0: link state changed to UP
re0: <Realtek PCIe GbE Family Controller> port 0xe000-0xe0ff mem 0xd0704000-0xd0704fff,0xd0700000-0xd0703fff irq 18 at device 0.0 on pci1
re0: Using Memory Mapping!
re0: Using 1 MSI-X message
re0: ASPM disabled
re0: version:1.98.00
re0: Ethernet address: d0:50:99:59:15:7e
re0: Ethernet address: d0:50:99:59:15:7e
re0: link state changed to UP

Second broken box:

$ ifconfig re0
re0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=201b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,WOL_MAGIC>
        ether 04:7c:16:44:25:0b
        inet 192.168.1.26 netmask 0xffffff00 broadcast 192.168.1.255
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>

# dmesg |grep re0
re0: <Realtek PCIe 2.5GbE Family Controller> port 0x4000-0x40ff mem 0x73d00000-0x73d0ffff,0x73d10000-0x73d13fff irq 17 at device 0.0 on pci4
re0: Using Memory Mapping!
re0: Using 1 MSI-X message
re0: ASPM disabled
re0: version:1.98.00
re0: Ethernet address: 04:7c:16:44:25:0b
re0: Ethernet address: 04:7c:16:44:25:0b
re0: link state changed to UP

Let me know if I can provide more info.

Kind regards.
Comment 1 iio7 2023-11-10 20:37:41 UTC
A little more info.

First broken box:

# pciconf -lcvb re0
re0@pci0:3:0:0: class=0x020000 rev=0x11 hdr=0x00 vendor=0x10ec device=0x8168 subvendor=0x1849 subdevice=0x8168
    vendor     = 'Realtek Semiconductor Co., Ltd.'
    device     = 'RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller'
    class      = network
    subclass   = ethernet
    bar   [10] = type I/O Port, range 32, base rxd000, size 256, enabled
    bar   [18] = type Memory, range 64, base rxd1804000, size 4096, enabled
    bar   [20] = type Prefetchable Memory, range 64, base rxd1800000, size 16384, enabled
    cap 01[40] = powerspec 3  supports D0 D1 D2 D3  current D0
    cap 05[50] = MSI supports 1 message, 64 bit 
    cap 10[70] = PCI-Express 2 endpoint MSI 1 max data 128(128)
                 max read 512
                 link x1(x1) speed 2.5(2.5) ASPM disabled(L0s/L1) ClockPM disabled
    cap 11[b0] = MSI-X supports 4 messages, enabled
                 Table in map 0x20[0x0], PBA in map 0x20[0x800]
    cap 03[d0] = VPD
    ecap 0001[100] = AER 1 0 fatal 0 non-fatal 3 corrected
    ecap 0002[140] = VC 1 max VC0
    ecap 0003[160] = Serial 1 01000000684ce000
    ecap 0018[170] = LTR 1

Second broken box:

# pciconf -lcvb re0
re0@pci0:4:0:0: class=0x020000 rev=0x05 hdr=0x00 vendor=0x10ec device=0x8125 subvendor=0x1462 subdevice=0x7d42
    vendor     = 'Realtek Semiconductor Co., Ltd.'
    device     = 'RTL8125 2.5GbE Controller'
    class      = network
    subclass   = ethernet
    bar   [10] = type I/O Port, range 32, base rx4000, size 256, enabled
    bar   [18] = type Memory, range 64, base rx73d00000, size 65536, enabled
    bar   [20] = type Memory, range 64, base rx73d10000, size 16384, enabled
    cap 01[40] = powerspec 3  supports D0 D1 D2 D3  current D0
    cap 05[50] = MSI supports 1 message, 64 bit, vector masks 
    cap 10[70] = PCI-Express 2 endpoint MSI 1 max data 256(256) RO
                 max read 4096
                 link x1(x1) speed 5.0(5.0) ASPM disabled(L0s/L1) ClockPM disabled
    cap 11[b0] = MSI-X supports 32 messages, enabled
                 Table in map 0x20[0x0], PBA in map 0x20[0x800]
    cap 03[d0] = VPD
    ecap 0001[100] = AER 2 0 fatal 0 non-fatal 1 corrected
    ecap 0002[148] = VC 1 max VC0
    ecap 0003[168] = Serial 1 01000000684ce000
    ecap 0017[178] = TPH Requester 1
    ecap 0018[204] = LTR 1
    ecap 001e[20c] = L1 PM Substates 1
    ecap 000b[21c] = Vendor [1] ID 0002 Rev 4 Length 256
Comment 2 iio7 2023-11-21 22:05:13 UTC
After upgrading to 14.0-RELEASE the problem persists. However, I have removed the realtek-re-kmod and gone back to the one in base where this isn't a problem. The original problem that made me switch no longer exist.
Comment 3 Alex Dupre freebsd_committer freebsd_triage 2023-11-22 11:24:27 UTC
Was the error the "Watchdog timeout" followed by a network reset?
Comment 4 iio7 2023-11-23 17:18:27 UTC
(In reply to Alex Dupre from comment #3)

Yes, exactly. From my old log:

Dec  8 01:01:41 foo kernel: re0: watchdog timeout
Dec  8 01:01:41 foo kernel: re0: link state changed to DOWN
Dec  8 01:01:45 foo kernel: re0: link state changed to UP
Dec  8 01:01:51 foo kernel: re0: watchdog timeout
Dec  8 01:01:51 foo kernel: re0: link state changed to DOWN
Dec  8 01:01:55 foo kernel: re0: link state changed to UP
Comment 5 Alex Dupre freebsd_committer freebsd_triage 2023-11-28 15:29:08 UTC
Can you try to manually build the 1.99 branch (https://github.com/alexdupre/rtl_bsd_drv/tree/v1.99) at the latest commit (9d48464) and provide an updated feedback on the SSL issue, please?
Comment 6 iio7 2023-12-16 01:45:51 UTC
(In reply to Alex Dupre from comment #5)

v1.99 has fixed the SSL issue!
Comment 7 Alex Dupre freebsd_committer freebsd_triage 2023-12-16 09:19:46 UTC
Great, thanks for testing.
Comment 8 iio7 2023-12-17 19:13:42 UTC
I think we can close this. Bug fixed in v1.99.