Bug 233952 - if_jme: Does not send/receive packets after 11.2 to 12.0 upgrade
Summary: if_jme: Does not send/receive packets after 11.2 to 12.0 upgrade
Status: Closed FIXED
Alias: None
Product: Base System
Classification: Unclassified
Component: kern (show other bugs)
Version: 12.0-RELEASE
Hardware: amd64 Any
: --- Affects Many People
Assignee: Mark Johnston
URL:
Keywords: regression
Depends on:
Blocks: 240700
  Show dependency treegraph
 
Reported: 2018-12-12 03:59 UTC by Hugh O'Brien
Modified: 2019-10-01 02:19 UTC (History)
5 users (show)

See Also:
koobs: maintainer-feedback? (yongari)
koobs: mfc-stable12+


Attachments
pciconf -vl and dmesg (13.32 KB, text/plain)
2018-12-12 03:59 UTC, Hugh O'Brien
no flags Details
dmesg.boot (8.20 KB, text/plain)
2019-09-17 14:41 UTC, Armin Gruner
no flags Details
pciconf -l output (8.88 KB, text/plain)
2019-09-17 14:41 UTC, Armin Gruner
no flags Details
Fixes if_jme regression under 12.0-RELEASE (2.92 KB, patch)
2019-09-21 15:21 UTC, Armin Gruner
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Hugh O'Brien 2018-12-12 03:59:04 UTC
Created attachment 200052 [details]
pciconf -vl and dmesg

jme device does not seem to send/receive packages after upgrading to 12.0-RELEASE.

CC: yongari@freebsd.org

tests:
 1. boot into 11.2-RELEASE live-cd, assign ip to jme device, ping local gw, success
 2. boot into 12.0-RELEASE live-cd, assign ip to jme device, ping local gw, failure


workaround:

disabling msi and msi_x allow the device to work again. I tested each separately but it seems they must both be disabled.

/boot/loader.conf:
hw.jme.msi_disable="1"
hw.jme.msix_disable="1"

FreeBSD vaio 12.0-RELEASE FreeBSD 12.0-RELEASE r341666 GENERIC  amd64
Comment 1 Hugh O'Brien 2018-12-12 04:01:56 UTC
Should have said: 
jme device does not seem to send/receive *PACKETS *after upgrading to 12.0-RELEASE.
Comment 2 Hugh O'Brien 2018-12-12 04:03:01 UTC
pciconf and dmesg output available in attachment
Comment 3 Armin Gruner 2019-09-11 15:38:00 UTC
I can confirm that;
I have a working 11.3-RELEASE installation on my ASUS EeeTop 1610,
and after installation of 12.0-RELAESE the jme0 interface does not receive packets anymore.

When checking with tcpdump on both ends, I see that ARP requests are received, and the remote 12.0-RELEASE box send an ARP reply back, but I never see that on the EeeTop. The running tcpdump on it only shows outgoing traffic, but no incoming. This is after a fresh installation without and tweaks, packages installed, enabled firewall et al.
Comment 4 Kubilay Kocak freebsd_committer freebsd_triage 2019-09-15 02:34:14 UTC
(In reply to Armin Gruner from comment #3)

Armin, could you please provide the following

- uname -a output
- /var/run/dmesg.boot output (as an attachment)
- pciconf -lv output (as an attachment)
Comment 5 Armin Gruner 2019-09-17 14:40:30 UTC
Sure:

root@eeetop:~ # uname -a
FreeBSD eeetop 12.0-RELEASE FreeBSD 12.0-RELEASE r341666 GENERIC  amd64

Output of dmesg after boot and pciconf is attached;
if I remove the disables from loader.conf, pciconf instead says
"enabled" in the PCI device capability 11 of device jme0. Then, DHCP waits forever, and I don't see incoming packets, as described earlier.

Cheers, Armin
Comment 6 Armin Gruner 2019-09-17 14:41:10 UTC
Created attachment 207576 [details]
dmesg.boot
Comment 7 Armin Gruner 2019-09-17 14:41:35 UTC
Created attachment 207577 [details]
pciconf -l output
Comment 8 Armin Gruner 2019-09-17 14:53:56 UTC
BTW I know how to compile kernels;
this box can be rebooted anytime, and I am happy to test snapshots (or build from 12-CURRENT, any git branch/tag, I sync with https://github.com/freebsd/freebsd.git

After a quick glance into the kernel sources I see that there are changes in sys/dev/jme between stable/11 and master, though the diff is minimal.
But the changes obviously affect programming the MSI/MSIX vectors..

If you like, I can just revert those changes and test with a newly built 12-HEAD kernel..
Comment 9 Armin Gruner 2019-09-17 15:49:00 UTC
> If you like, I can just revert those changes and test with a newly built 12-HEAD kernel..

I answer myself:
If I revert the changes in if_jme.c introduced between 11.3-RELEASE and 12.0-RELEASE,  the jme0 ethernet interface is working again in 12.0-RELEASE
Comment 10 Mark Johnston freebsd_committer freebsd_triage 2019-09-17 16:08:42 UTC
(In reply to Armin Gruner from comment #9)
Can you narrow this down to a specific revision?
Comment 11 Armin Gruner 2019-09-17 18:36:45 UTC
https://svnweb.freebsd.org/base?view=revision&revision=316820

breaks it
Comment 12 Mark Johnston freebsd_committer freebsd_triage 2019-09-17 20:51:43 UTC
(In reply to Armin Gruner from comment #11)
I can't see a problem in that commit, and I haven't been able to find a programmer reference for the jmicron controllers with a quick search.  Linux doesn't attempt to map the interrupt sources at all.  I suspect that we should simply revert the change for 12.1.
Comment 13 Armin Gruner 2019-09-19 20:50:33 UTC
I think that's the best resolution, perhaps the committer can bring any light into darkness? -> yongari
Comment 14 Kubilay Kocak freebsd_committer freebsd_triage 2019-09-20 04:39:55 UTC
With 12.1 imminent, I'd recommend preparing a patch (differential) for review as soon as possible, if reverting base r316820 is confirmed to resolve the issue

@Armin, can you test reverting that patch, rebuilding your kernel (or just if_jme) and explicitly confirm that it resolves the issue?
Comment 15 Hugh O'Brien 2019-09-20 09:40:58 UTC
I can also test any patches.
Comment 16 Kubilay Kocak freebsd_committer freebsd_triage 2019-09-20 10:12:17 UTC
(In reply to Hugh O'Brien from comment #15)

The patch is a reversion of base r316820
Comment 17 Armin Gruner 2019-09-21 15:20:03 UTC
Attached reverse patch fixes the problem, GENERIC compiles (under 12.0-RELEASE)

--------------------------------------------------------------
>>> Kernel build for GENERIC completed on Sat Sep 21 17:13:53 CEST 2019
--------------------------------------------------------------
Comment 18 Armin Gruner 2019-09-21 15:21:25 UTC
Created attachment 207686 [details]
Fixes if_jme regression under 12.0-RELEASE
Comment 19 commit-hook freebsd_committer freebsd_triage 2019-09-23 14:29:13 UTC
A commit references this bug:

Author: markj
Date: Mon Sep 23 14:29:05 UTC 2019
New revision: 352625
URL: https://svnweb.freebsd.org/changeset/base/352625

Log:
  Revert r316820.

  Despite appearing correct, r316820 breaks packet rx/tx for jme(4)
  interfaces.  With 12.1 approaching, let's just revert the commit for now.

  PR:		233952
  Tested by:	Armin Gruner <ag-freebsd@muc.de>
  MFC after:	3 days

Changes:
  head/sys/dev/jme/if_jme.c
Comment 20 Kubilay Kocak freebsd_committer freebsd_triage 2019-09-23 14:33:45 UTC
Add to 12.1-RELEASE blockers given comment 19
Comment 21 commit-hook freebsd_committer freebsd_triage 2019-09-26 15:38:22 UTC
A commit references this bug:

Author: markj
Date: Thu Sep 26 15:37:40 UTC 2019
New revision: 352749
URL: https://svnweb.freebsd.org/changeset/base/352749

Log:
  MFC r352625:
  Revert r316820.

  PR:	233952

Changes:
_U  stable/12/
  stable/12/sys/dev/jme/if_jme.c
Comment 22 commit-hook freebsd_committer freebsd_triage 2019-09-26 18:36:52 UTC
A commit references this bug:

Author: markj
Date: Thu Sep 26 18:36:15 UTC 2019
New revision: 352766
URL: https://svnweb.freebsd.org/changeset/base/352766

Log:
  MFS r352749:
  Revert r316820.

  PR:		233952
  Approved by:	re (gjb)

Changes:
_U  releng/12.1/
  releng/12.1/sys/dev/jme/if_jme.c
Comment 23 Mark Johnston freebsd_committer freebsd_triage 2019-09-26 18:38:57 UTC
This will be fixed in 12.1.  Thanks for the report and for tracking down the commit responsible.