Bug 263091 - [genet] genet interface is not able to transmit/receive packet after the interface is marked "down" then marked "up"
Summary: [genet] genet interface is not able to transmit/receive packet after the inte...
Status: Closed FIXED
Alias: None
Product: Base System
Classification: Unclassified
Component: bin (show other bugs)
Version: CURRENT
Hardware: arm64 Any
: --- Affects Some People
Assignee: Mike Karels
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2022-04-06 14:05 UTC by Jiahao LI
Modified: 2022-07-26 12:41 UTC (History)
4 users (show)

See Also:


Attachments
preliminary patch (1.52 KB, patch)
2022-04-06 23:52 UTC, Mike Karels
no flags Details | Diff
Bug fix batch of genet interface down (2.17 KB, patch)
2022-04-08 18:38 UTC, Jiahao LI
no flags Details | Diff
hybrid patch (3.60 KB, patch)
2022-04-09 15:06 UTC, Mike Karels
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Jiahao LI 2022-04-06 14:05:03 UTC
Overview
--------

The genet interface is not able transmit/receive any packets after the genet interface is marked "down"
firstly then marked "up".

The reason is that the implementation genet driver code will reset the genet interface but not change the runtime flag of genet driver. Therefore, when "ifconfig genet up" command is used, the genet driver cannot be re-initialized again. Moreover, the initialization of genet driver is to enbale the driver's DMA and interrupts, which is not enbale the genet driver being reset when genet is marked down.

Steps to Reproduce
------------------

1. Config a static IP address to both of the host's interface and the genet interface

In the DUT(Device Under Test), with genet interface and Freebsd running, side

root@generic:~ # ifconfig genet0 192.168.3.129 netmask 255.255.255.0
root@generic:~ # ifconfig genet0
genet0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=68000b<RXCSUM,TXCSUM,VLAN_MTU,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
        ether dc:a6:32:bf:f4:e6
        inet 192.168.3.129 netmask 0xffffff00 broadcast 192.168.3.255
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>

        
In the host computer with direct Ethernet cable connection to the DUT

$ sudo ifconfig enp0s31f6 192.168.3.2
$ ifconfig enp0s31f6
enp0s31f6: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.3.2  netmask 255.255.255.0  broadcast 192.168.3.255
        ether 8c:8c:aa:c1:2b:c3  txqueuelen 1000  (Ethernet)
        RX packets 384  bytes 37406 (37.4 KB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 5674  bytes 1175798 (1.1 MB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
        device interrupt 16  memory 0xae380000-ae3a0000
        
2. Host's interface and genet interface can transmit/receive packet to/from each other

In the DUT

root@generic:~ # ping 192.168.3.2
PING 192.168.3.2 (192.168.3.2): 56 data bytes
64 bytes from 192.168.3.2: icmp_seq=0 ttl=64 time=0.445 ms
64 bytes from 192.168.3.2: icmp_seq=1 ttl=64 time=0.321 ms
64 bytes from 192.168.3.2: icmp_seq=2 ttl=64 time=0.361 ms
^C
--- 192.168.3.2 ping statistics ---
3 packets transmitted, 3 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 0.321/0.376/0.445/0.051 ms

In the host

$ ping 192.168.3.129
PING 192.168.3.129 (192.168.3.129) 56(84) bytes of data.
64 bytes from 192.168.3.129: icmp_seq=1 ttl=64 time=0.242 ms
64 bytes from 192.168.3.129: icmp_seq=2 ttl=64 time=0.233 ms
64 bytes from 192.168.3.129: icmp_seq=3 ttl=64 time=0.180 ms
^C
--- 192.168.3.129 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2049ms
rtt min/avg/max/mdev = 0.180/0.218/0.242/0.027 ms

3. Mark the genet interface "down", then mark the interface "up"

root@generic:~ # ifconfig genet0 down
root@generic:~ # ifconfig genet0
genet0: flags=8842<BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=68000b<RXCSUM,TXCSUM,VLAN_MTU,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
        ether dc:a6:32:bf:f4:e6
        inet 192.168.3.129 netmask 0xffffff00 broadcast 192.168.3.255
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
root@generic:~ # ifconfig genet0 up
root@generic:~ # ifconfig genet0 
genet0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=68000b<RXCSUM,TXCSUM,VLAN_MTU,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
        ether dc:a6:32:bf:f4:e6
        inet 192.168.3.129 netmask 0xffffff00 broadcast 192.168.3.255
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>

Actual Results
--------------

1. Genet interface is not able to transmit/receive any packet

In the DUT

root@generic:~ # ping 192.168.3.2
PING 192.168.3.2 (192.168.3.2): 56 data bytes
^C
--- 192.168.3.2 ping statistics ---
4 packets transmitted, 0 packets received, 100.0% packet loss

In the host

$ ping 192.168.3.129
PING 192.168.3.129 (192.168.3.129) 56(84) bytes of data.
From 192.168.3.2 icmp_seq=9 Destination Host Unreachable
From 192.168.3.2 icmp_seq=10 Destination Host Unreachable
From 192.168.3.2 icmp_seq=11 Destination Host Unreachable
^C
--- 192.168.3.129 ping statistics ---
13 packets transmitted, 0 received, +3 errors, 100% packet loss, time 12280ms
pipe 4

Expected Results
----------------
The "ifconfig genet0 up" can re-enable the genet inerface after the genet interface is marked as "down". The data communication between the genet and the host should be recovered when the interface is re-enbale.

Build Date & Hardware
---------------------
root@generic:~ # uname -a
FreeBSD generic 14.0-CURRENT FreeBSD 14.0-CURRENT #0 main-n254105-d53927b0bae: Thu Mar 31 09:26:32 UTC 2022     root@releng1.nyi.freebsd.org:/usr/obj/usr/src/arm64.aarch64/sys/GENERIC arm64
Comment 1 Mike Karels freebsd_committer freebsd_triage 2022-04-06 20:19:16 UTC
Thanks for the report.  I have reproduced this, and am looking into it.
Comment 2 Mike Karels freebsd_committer freebsd_triage 2022-04-06 23:52:06 UTC
Created attachment 233010 [details]
preliminary patch

Can you try this patch to src/sys/arm64/broadcom/genet/if_genet.c?  It seems to work.
Comment 3 Jiahao LI 2022-04-08 18:38:29 UTC
Created attachment 233077 [details]
Bug fix batch of genet interface down

A copy of the bug fix patch which referred the implementation of Openbsd code.
Comment 4 Jiahao LI 2022-04-08 18:38:58 UTC
(In reply to Mike Karels from comment #2)
Hi Mike,

Thanks for the patch.

I tried your patch in my board. It seems like it is not working in my rpi4. Iam wondering if it is working in your board. Hoping I am not missing any things.

I have also tried fixed this bug before. My patch is also attached, which is referred from the implementation from the OpenBsd. It is working in my board. I attach a copy of the bug fix patch from our internal repo.

Jiahao
Comment 5 Mike Karels freebsd_committer freebsd_triage 2022-04-09 15:06:59 UTC
Created attachment 233100 [details]
hybrid patch

Thanks for your patch.  My earlier patch worked on my RPi4, which is an original model with 4 GB.  Your patch is more complete, which would also be useful if someone turns this into a module.  I hybridized the two, as I wanted the full reset done by gen_reset; some of the other code may not be needed with that, but I left it.  Could you try this one?  Thanks.
Comment 6 Jiahao LI 2022-04-14 17:40:50 UTC
(In reply to Mike Karels from comment #5)
Hi Mike,

Sorry for the late reply.

The new hybrid patch works well in my rpi4 model B 8G RAM board.

Thanks,
Jiahao
Comment 7 commit-hook freebsd_committer freebsd_triage 2022-04-14 19:11:18 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=8f45652b6bccc258eb58f8721dea10184f1aaa17

commit 8f45652b6bccc258eb58f8721dea10184f1aaa17
Author:     Mike Karels <karels@FreeBSD.org>
AuthorDate: 2022-04-11 19:44:49 +0000
Commit:     Mike Karels <karels@FreeBSD.org>
CommitDate: 2022-04-14 19:10:13 +0000

    genet: fix problems with interface down/up

    The genet interface did not resume operation correctly after doing
    ifconfig down then up.  The down/reset procedure did not clear the
    RUNNING flag, and did not reset enough of the hardware state.  This
    patch is modeled on OpenBSD code, with a call to gen_reset added
    to reset the controller completely.  Regularize the parameter to
    gen_dma_disable() while here.

    PR:             263091
    Submitted by:   jiahali@blackberry.com

 sys/arm64/broadcom/genet/if_genet.c | 86 ++++++++++++++++++++++++++++++++-----
 1 file changed, 75 insertions(+), 11 deletions(-)
Comment 8 Daniel O'Connor 2022-07-26 11:57:41 UTC
Any chance this can be MFCd before 13.2?
Comment 9 Mike Karels freebsd_committer freebsd_triage 2022-07-26 12:15:35 UTC
(In reply to Daniel O'Connor from comment #8)
The change has been on stable/13 since April, so will automatically be included in 13.2.
Comment 10 Daniel O'Connor 2022-07-26 12:41:53 UTC
Thanks! (Sorry for the noise, I checked the wrong branch..)