Bug 208343

Summary: [em] wake on lan not working with Intel I219 V2
Product: Base System Reporter: Guido Falsi <madpilot>
Component: kernAssignee: Sean Bruno <sbruno>
Status: Closed FIXED    
Severity: Affects Some People CC: andreas.schroden, dengxf, erj, jeffrey.e.pieper, kaho, marius, naito.yuichiro, sbruno, vangyzen, wblock, weinrank
Priority: --- Keywords: IntelNetworking
Version: CURRENTFlags: koobs: mfc-stable11?
koobs: mfc-stable10?
Hardware: amd64   
OS: Any   
Attachments:
Description Flags
pciconf -lv output
none
a very initial patch to enable WOL for igb
none
2nd patch for igb
none
IFLIB update for WOL issues on igb(4) devices
none
IFLIB update for igb, 2nd
none
WOL Support for igb(4) on stable/11
none
MFC r322986 to stable/11 none

Description Guido Falsi freebsd_committer freebsd_triage 2016-03-27 23:28:36 UTC
Created attachment 168707 [details]
pciconf -lv output

As subject states with the mentioned hardware Wake on LAN is not working properly after shutting down the system from FreeBSD. It does work fine if powering down the machine from the BIOS.

I have an Asus H170M-PLUS Mother Board.

Also the WOL capabilities are never listed in ifconfig and cannot be activated:

# ifconfig em0
em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500

options=4019b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,VLAN_HWTSO>
        ether XXX
        inet XXX netmask 0xffffff00 broadcast XXX
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
        media: Ethernet autoselect (100baseTX <full-duplex>)
        status: active
# ifconfig em0 wol
# ifconfig em0
em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500

options=4019b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,VLAN_HWTSO>
        ether XXX
        inet XXX netmask 0xffffff00 broadcast XXX
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
        media: Ethernet autoselect (100baseTX <full-duplex>)
        status: active
# sysctl dev.em.0.wake=1
dev.em.0.wake: 0 -> 1
# ifconfig em0
em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500

options=4019b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,VLAN_HWTSO>
        ether XXX
        inet XXX netmask 0xffffff00 broadcast XXX
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
        media: Ethernet autoselect (100baseTX <full-duplex>)
        status: active
# ifconfig em0 wol
# ifconfig em0
em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500

options=4019b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,VLAN_HWTSO>
        ether XXX
        inet XXX netmask 0xffffff00 broadcast XXX
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
        media: Ethernet autoselect (100baseTX <full-duplex>)
        status: active
#


Attaching output of pciconf -lv. If any other information is needed please ask!

Thanks!


I'm also linking the ML thread for reference:
https://lists.freebsd.org/pipermail/freebsd-net/2016-March/044913.html
Comment 1 Guido Falsi freebsd_committer freebsd_triage 2016-03-27 23:43:33 UTC
I forgot to add the following information:

FreeBSD marvin.madpilot.net 11.0-CURRENT FreeBSD 11.0-CURRENT #132 r297146M: Thu Mar 24 18:36:08 CET 2016     root@marvin.madpilot.net:/usr/obj/usr/src/sys/MARVIN  amd64

(the sources include a patch related to the VFS I'm testing, unrelated to the network stack)
Comment 2 Guido Falsi freebsd_committer freebsd_triage 2016-03-27 23:46:10 UTC
Oops, I was too fast, the correct uname is this one:

FreeBSD bender.madpilot.net 11.0-CURRENT FreeBSD 11.0-CURRENT #3 r297146M: Mon Mar 21 11:20:59 CET 2016     root@bender.madpilot.net:/usr/obj/usr/src/sys/BENDER  amd64

same sources though.
Comment 3 Sean Bruno freebsd_committer freebsd_triage 2016-03-27 23:48:07 UTC
btw, what do you use to send the WOL packet to the machine and what's the command line you are using?
Comment 4 Eric van Gyzen freebsd_committer freebsd_triage 2016-03-28 00:17:16 UTC
I get the same behavior as Guido.  I'm running head r297219 (24 March) on a Dell XPS 8900 with an onboard NIC.  To send the WOL packet, I run this on a neighboring 10.2-RELEASE machine:

    wake bridge0 64:00:6a:80:be:cd

As I mentioned in the mail thread:  In em_get_wakeup(), the eeprom_data is zero, so adapter->wol stays zero, so ifconfig doesn't show any WOL capabilities.  When I ignore the eeprom_data by commenting out the "if" condition, WOL capabilities appear in ifconfig.  However, even when I enable WOL with ifconfig, WOL still doesn't work after a "shutdown -p now".


em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
	options=4219b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_MAGIC,VLAN_HWTSO>
	ether 64:00:6a:80:be:cd
	inet a.b.c.d netmask 0xffffff80 broadcast e.f.g.h
	nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
	media: Ethernet autoselect (1000baseT <full-duplex>)
	status: active

em0@pci0:0:31:6:	class=0x020000 card=0x06b81028 chip=0x15b88086 rev=0x31 hdr=0x00
    vendor     = 'Intel Corporation'
    device     = 'Ethernet Connection (2) I219-V'
    class      = network
    subclass   = ethernet

smbios.system.maker="Dell Inc."
smbios.system.product="XPS 8900"
smbios.bios.reldate="11/12/2015"
smbios.bios.version="2.1.0"
Comment 5 Guido Falsi freebsd_committer freebsd_triage 2016-03-28 08:46:48 UTC
(In reply to Sean Bruno from comment #3)
> btw, what do you use to send the WOL packet to the machine and what's the
> command line you are using?

I'm using the wake command included in FreeBSD, same command line Eric is using:

wake bridge0 bender.madpilot.net

as root, bender.madpilot.net is listed in /etc/ethers obviously.
Comment 6 Guido Falsi freebsd_committer freebsd_triage 2016-06-29 12:30:07 UTC
Has there been any progress related to this issue? Any chance to get a fix in time to have it included in 11.0?

Thanks in advance.
Comment 7 Warren Block freebsd_committer freebsd_triage 2016-10-20 14:15:25 UTC
Seeing the same symptoms with I217-LM (class=0x020000 card=0x153a15d9 chip=0x153a8086 rev=0x05 hdr=0x00) and I210 (card=0x153315d9 chip=0x15338086 rev=0x03 hdr=0x00) interfaces on Supermicro X10SLL-F boards.  The WOL capabilities do not appear in the list, and the system does not wake.  Supermicro support says that WOL is enabled on these boards.  There are no WOL options in the UEFI settings.
Comment 8 Sean Bruno freebsd_committer freebsd_triage 2016-11-07 13:48:24 UTC
Can you guys try the following and see if it helps?

Index: if_em.c
===================================================================
--- if_em.c    (revision 308404)
+++ if_em.c    (working copy)
@@ -5274,6 +5274,8 @@ em_get_wakeup(device_t dev)
     case e1000_ich10lan:
     case e1000_pchlan:
     case e1000_pch2lan:
+    case e1000_pch_lpt:
+    case e1000_pch_spt:
         apme_mask = E1000_WUC_APME;
         adapter->has_amt = TRUE;
         eeprom_data = E1000_READ_REG(&adapter->hw, E1000_WUC);
Comment 9 Eric van Gyzen freebsd_committer freebsd_triage 2016-11-07 14:36:23 UTC
(In reply to Sean Bruno from comment #8)

That doesn't work, but I think it's part of the fix:  Before the patch, when I powered off the system from FreeBSD, the NIC link/activity LEDs stayed off.  With the patch, the link LED stays on and the activity LED blinks as expected.

Other folks:  Please test the patch, in case it really works but I've misconfigured something in my BIOS.
Comment 10 Sean Bruno freebsd_committer freebsd_triage 2016-11-07 15:03:14 UTC
I wonder if we're going to need a variation on this linux kernel commit:

https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/drivers/net/ethernet/intel/e1000e?id=6607c99e7034e7565a1559a24dd35083d6719788
Comment 11 Guido Falsi freebsd_committer freebsd_triage 2016-11-07 22:02:19 UTC
(In reply to Sean Bruno from comment #8)
> Can you guys try the following and see if it helps?

At a first test I see similar behaviour as reported by Eric.

Network interface blinking, but no wake up.

The output from ifconfig also now shows WOL features:

em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=4219b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_MAGIC,VLAN_HWTSO>
	ether XXX
	inet XXX netmask 0xffffff00 broadcast XXX
	nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
	media: Ethernet autoselect (100baseTX <full-duplex>)
	status: active
Comment 12 Eric van Gyzen freebsd_committer freebsd_triage 2016-11-07 22:03:55 UTC
I forgot to mention:  ifconfig showed WOL features for me, too.  WOL_MAGIC was enabled.
Comment 13 Kaho Toshikazu 2016-11-09 10:19:39 UTC
Can you try this patch? I am not sure it is a correct way.

Index: sys/dev/e1000/if_em.c
===================================================================
--- sys/dev/e1000/if_em.c       (revision 308444)
+++ sys/dev/e1000/if_em.c       (working copy)
@@ -5274,6 +5274,8 @@
        case e1000_ich10lan:
        case e1000_pchlan:
        case e1000_pch2lan:
+       case e1000_pch_lpt:
+       case e1000_pch_spt:
                apme_mask = E1000_WUC_APME;
                adapter->has_amt = TRUE;
                eeprom_data = E1000_READ_REG(&adapter->hw, E1000_WUC);
@@ -5322,7 +5324,7 @@
 {
        struct adapter  *adapter = device_get_softc(dev);
        if_t ifp = adapter->ifp;
-       u32             pmc, ctrl, ctrl_ext, rctl;
+       u32             pmc, ctrl, wuc, ctrl_ext, rctl;
        u16             status;
 
        if ((pci_find_cap(dev, PCIY_PMG, &pmc) != 0))
@@ -5332,7 +5334,9 @@
        ctrl = E1000_READ_REG(&adapter->hw, E1000_CTRL);
        ctrl |= (E1000_CTRL_SWDPIN2 | E1000_CTRL_SWDPIN3);
        E1000_WRITE_REG(&adapter->hw, E1000_CTRL, ctrl);
-       E1000_WRITE_REG(&adapter->hw, E1000_WUC, E1000_WUC_PME_EN);
+       wuc = E1000_READ_REG(&adapter->hw, E1000_WUC);
+       wuc |= E1000_WUC_PME_EN ;
+       E1000_WRITE_REG(&adapter->hw, E1000_WUC, wuc);
 
        if ((adapter->hw.mac.type == e1000_ich8lan) ||
            (adapter->hw.mac.type == e1000_pchlan) ||
@@ -5364,7 +5368,9 @@
        }
 
        if ((adapter->hw.mac.type == e1000_pchlan) ||
-           (adapter->hw.mac.type == e1000_pch2lan)) {
+           (adapter->hw.mac.type == e1000_pch2lan) ||
+           (adapter->hw.mac.type == e1000_pch_lpt) ||
+           (adapter->hw.mac.type == e1000_pch_spt)) {
                if (em_enable_phy_wakeup(adapter))
                        return;
        } else {
Comment 14 Guido Falsi freebsd_committer freebsd_triage 2016-11-09 12:10:25 UTC
(In reply to Kaho Toshikazu from comment #13)
> Can you try this patch? I am not sure it is a correct way.

This one worked.

Applied patch, rebuilt and installed kernel, rebooted, shut down sent WOL packet and it went up like a charm!

Thanks.

If you need further testing, just ask!
Comment 15 Sean Bruno freebsd_committer freebsd_triage 2016-11-09 20:51:25 UTC
(In reply to Kaho Toshikazu from comment #13)
this looks good to me.  I will commit if there are no objections from the maintainers.
Comment 16 commit-hook freebsd_committer freebsd_triage 2016-11-14 17:19:15 UTC
A commit references this bug:

Author: sbruno
Date: Mon Nov 14 17:19:04 UTC 2016
New revision: 308643
URL: https://svnweb.freebsd.org/changeset/base/308643

Log:
  Update WOL support for newer em(4) devices.

  Do not overwrite the contents of the WUC register, add E1000_WUC_PME_EN
  to the register contents, leaving the default contents intact.

  PR:		208343
  Submitted by:	Kaho Toshikazu <kaho@elam.kais.kyoto-u.ac.jp>
  Reviewed by:	jeffrey piper <jeffrey.e.pieper@intel.com>
  Approved by:	erj@
  MFC after:	2 weeks

Changes:
  head/sys/dev/e1000/if_em.c
Comment 17 Warren Block freebsd_committer freebsd_triage 2016-11-15 02:59:02 UTC
The em0 interface on the Supermicro X10SLL-F board mentioned in comment #7 shows a WOL capability and does wake the machine now.  The igb0 interface does not.  Could a similar fix apply to it?
Comment 18 Sean Bruno freebsd_committer freebsd_triage 2016-11-15 04:18:38 UTC
(In reply to Warren Block from comment #17)
Probably similar, but I'm unsure based on the fix in this ticket.

I'd suggest spamming a new ticket with pciconv -lv output against igb(4) and lets see what we can see.
Comment 19 Jeff Pieper 2016-11-15 11:59:43 UTC
Generally for LOMs, the BIOS/FW will only allow WOL to be active on one interface. That behavior is expected imo.
Comment 20 Kaho Toshikazu 2016-11-16 10:58:00 UTC
Created attachment 177062 [details]
a very initial patch to enable WOL for igb

> The igb0 interface does not.  Could a similar fix apply to it?

It seems different problems exist. 
The patch attached with this comment forces to enable WOL. 
Its features are below here.

1. The WOL bit in the variable if_capabilities is always set.
2. Add a functionality for ifconfig wol options.
3. Clear WUFC register instead of WFC register on a reset time.
Comment 21 Sean Bruno freebsd_committer freebsd_triage 2016-11-16 15:07:28 UTC
(In reply to Kaho Toshikazu from comment #20)
I think it is ok to *delete* the code being moved instead of #if 0 after testing confirms this as working.
Comment 22 Kaho Toshikazu 2016-11-17 01:31:22 UTC
Created attachment 177103 [details]
2nd patch for igb

comment #21
Thank you for your comment.

I deleted #if 0 and generated 2nd patch. WOL capabilities is always set,
but capenable depends on the contents of eeprom.

--
If try this patch, don't forget to check WOL is enabled.
When any WOL options are not enabled, please execute `ifconfig igb0 wol_magic`
before shutdown your machine.
Comment 23 Sean Bruno freebsd_committer freebsd_triage 2016-11-17 01:47:55 UTC
(In reply to Kaho Toshikazu from comment #22)
Thank you for working on this.  Let's see if we can get some folks testing various igb(4) devices for success.
Comment 24 Warren Block freebsd_committer freebsd_triage 2016-11-21 19:36:40 UTC
Patch applied, but WOL_MAGIC not appearing for igb0:

% ifconfig
em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 6122
options=4219b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_MAGIC,VLAN_HWTSO>
	ether 0c:c4:7a:90:5f:6f
	inet 10.0.0.12 netmask 0xffffff00 broadcast 10.0.0.255 
	nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
	media: Ethernet autoselect (1000baseT <full-duplex>)
	status: active
igb0: flags=8c02<BROADCAST,OACTIVE,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=6403bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6>
	ether 0c:c4:7a:90:5f:6e
	nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
	media: Ethernet autoselect
	status: no carrier

It was mentioned earlier that having WOL enabled on one interface might prevent it on others.  This motherboard has no options in the setup screens for WOL at all.
Comment 25 Warren Block freebsd_committer freebsd_triage 2016-11-21 20:08:09 UTC
Running 'ifconfig igb0 wol_magic' produced no errors, but the machine does not wake up when a WOL packet is sent to that interface.

For what it's worth, the em0 interface shows both LEDs on solid, amber and green, when the system is off but can be woken.  Only the amber LED is lit on the igb0 in the same state (same cable, same switch).
Comment 26 Sean Bruno freebsd_committer freebsd_triage 2016-11-21 21:38:07 UTC
(In reply to Warren Block from comment #25)
I seem to recall that the Intel ROM itself has a knob in it for WOL.

Can you access this menu during the bootup and see if its a thing?
Comment 27 Kaho Toshikazu 2016-11-22 02:20:22 UTC
(In reply to Warren Block from comment #25)

Thank you for your reports. 

Are you testing with staying the interface down? Can you try to up
the interface at first? The command line is like this
`ifconfig igb0 up wol_magic`.

The igb driver does not touch receive control register (E1000_RCTL)
at enabling WOL.
Comment 28 Warren Block freebsd_committer freebsd_triage 2016-11-22 03:59:23 UTC
Both the em0 and igb0 interfaces are built into this motherboard, not add-on cards.  There is no separate menu for either interface, at least not obviously.  It might be possible to change some settings with ipmitool.

`ifconfig igb0 up wol_magic` does make it work!  WOL_MAGIC is not set by default, but after setting it and making sure the interface is up, it does wake.

Thanks!
Comment 29 Guido Falsi freebsd_committer freebsd_triage 2016-11-28 12:46:58 UTC
Hi,

Since I submitted this bug I'm asking what is the policy for accounting it as fixed?

The bug I reported has been fixed in r308643, so my opinion is this bug report could be closed as fixed.

Should I close it, or wait for a src committer to close it? Should I wait for the MFC or the other commenters to report back?
Comment 30 Kubilay Kocak freebsd_committer freebsd_triage 2016-11-28 12:54:00 UTC
(In reply to Guido Falsi from comment #29)

1) Assign to committer resolving (sbruno)

Note: Committers should assign themselves as early as possible and reference PR: in commit log messages.

2) Set mfc-stableX flags to ? (means request/ask for mfc)

Committer resolving will either:

a) Set + upon MFC to that branch, or
b) Set - with comment outlining why not (unnecessary, breaks kbi, etc)
Comment 31 Kubilay Kocak freebsd_committer freebsd_triage 2016-11-28 12:54:54 UTC
3) Assignee will then close, sometimes pending confirmation from original Reporter :)
Comment 32 commit-hook freebsd_committer freebsd_triage 2017-01-12 14:48:28 UTC
A commit references this bug:

Author: sbruno
Date: Thu Jan 12 14:47:44 UTC 2017
New revision: 311982
URL: https://svnweb.freebsd.org/changeset/base/311982

Log:
  Restore fixup for newer em(4) devices WOL capabilities post iflib integration.

  PR:	208343

Changes:
  head/sys/dev/e1000/if_em.c
Comment 33 Sean Bruno freebsd_committer freebsd_triage 2017-01-12 16:37:59 UTC
Created attachment 178797 [details]
IFLIB update for WOL issues on igb(4) devices

Can I get this patch tested to see if it handles WOL issues on IGB devices now that we've converted igb(4) to the IFLIB framework?
Comment 34 Guido Falsi freebsd_committer freebsd_triage 2017-01-12 22:22:08 UTC
(In reply to Sean Bruno from comment #33)
> Created attachment 178797 [details]
> IFLIB update for WOL issues on igb(4) devices
> 
> Can I get this patch tested to see if it handles WOL issues on IGB devices
> now that we've converted igb(4) to the IFLIB framework?

I'll be able to test this next week. I'm away from home right now.
Comment 35 Kaho Toshikazu 2017-01-13 09:11:52 UTC
Created attachment 178851 [details]
IFLIB update for igb, 2nd

(In reply to Sean Bruno from comment #33)

Thank you for making the patch for igb devices.
I upload more simple patch.

What are changed:
1. I think that newer em and all igb devices have almost same
   WOL functions. All igb devices are treating as newer em.

2. Don't clear all bits of E1000_WUC register, especially
   clearing E1000_WUC_ASME bit makes problem.  I think that
   clearing E1000_WUC register is incorrect. It should clear
   E1000_WUFC register.

3. It may be a IFLIB problem. When a device are attached, its
   ifcapability and ifcapenable setup same value. Old driver can
   be having a function disabled but capable. IFLIB driver
   enables all capable fuctions. Now, the codes probing WOL
   capabilities are almost meaningless. All devices setup as WOL
   capable and all WOL function are enabled. Then, the code of
   doing enable should be change to doing disable.

4. WOL_UCAST bit makes being able to change.
Comment 36 Guido Falsi freebsd_committer freebsd_triage 2017-01-15 21:15:46 UTC
(In reply to Kaho Toshikazu from comment #35)
> Created attachment 178851 [details]
> IFLIB update for igb, 2nd
> 
> (In reply to Sean Bruno from comment #33)
> 
> Thank you for making the patch for igb devices.
> I upload more simple patch.
> 

I tested this patch applied to r312076 and it works, no regressions observed.
Comment 37 Warren Block freebsd_committer freebsd_triage 2017-01-18 19:46:32 UTC
Working here.  Thanks!
Comment 38 Sean Bruno freebsd_committer freebsd_triage 2017-01-18 20:07:35 UTC
(In reply to Warren Block from comment #37)
ok, I'll update at some point today after I'm done playing with EARLY_AP_STARTUP problems.
Comment 39 commit-hook freebsd_committer freebsd_triage 2017-01-22 18:05:47 UTC
A commit references this bug:

Author: sbruno
Date: Sun Jan 22 18:04:57 UTC 2017
New revision: 312641
URL: https://svnweb.freebsd.org/changeset/base/312641

Log:
  igb(4) enable WOL features for this class of devices.

  PR:	208343
  Submitted by:	Kaho Tashikazu <kaho@elam.kais.kyoto-u.ac.jp>

Changes:
  head/sys/dev/e1000/if_em.c
Comment 40 Xuefeng deng 2017-03-09 13:49:04 UTC
stable/11 still not have above patch, can someone merge them from current?
Thanks
Comment 41 Sean Bruno freebsd_committer freebsd_triage 2017-04-05 20:25:35 UTC
Created attachment 181517 [details]
WOL Support for igb(4) on stable/11

This needs testing.  I'm not sure if I've hit all the bits here.  Please test on stable/11 and report back.
Comment 42 Yuichiro NAITO 2017-05-02 01:44:50 UTC
I patched r311982 to my FreeBSD-11.0-RELEASE-p10 kernel.
em driver works for WOL_MAGIC on I219-LM.

I hope r311982 will be merged to stable branch.
Comment 43 commit-hook freebsd_committer freebsd_triage 2017-08-27 18:18:18 UTC
A commit references this bug:

Author: marius
Date: Sun Aug 27 18:18:01 UTC 2017
New revision: 322955
URL: https://svnweb.freebsd.org/changeset/base/322955

Log:
  MFC: r308643, r312427

  - Update WOL support for newer em(4) devices. [1]
  - Add support for Kaby Lake generation i219 (4) and i219 (5) devices.

  PR:	208343 [1]

Changes:
_U  stable/11/
  stable/11/sys/dev/e1000/e1000_82575.c
  stable/11/sys/dev/e1000/e1000_82575.h
  stable/11/sys/dev/e1000/e1000_defines.h
  stable/11/sys/dev/e1000/e1000_ich8lan.c
  stable/11/sys/dev/e1000/if_em.c
Comment 44 commit-hook freebsd_committer freebsd_triage 2017-08-27 21:37:05 UTC
A commit references this bug:

Author: marius
Date: Sun Aug 27 21:36:44 UTC 2017
New revision: 322956
URL: https://svnweb.freebsd.org/changeset/base/322956

Log:
  MFC: r312641

  Enable WOL features also for igb(4) class of devices.

  PR:		208343
  Submitted by:	Kaho Tashikazu <kaho@elam.kais.kyoto-u.ac.jp>

Changes:
_U  stable/11/
  stable/11/sys/dev/e1000/if_igb.c
Comment 45 Xuefeng deng 2017-08-28 12:03:15 UTC
I got a new issue with today's build. when I "shutdown -p" the machine,  it will auto-boot after a few seconds.

my nic is I219-V in a ASUS Z270-AR motherboard:

em0@pci0:0:31:6:        class=0x020000 card=0x86721043 chip=0x15b88086 rev=0x00 hdr=0x00                                                                                                                                                                                                                                     
    vendor     = 'Intel Corporation'                                                                                                                                                                                                                                                                                         
    device     = 'Ethernet Connection (2) I219-V'                                                                                                                                                                                                                                                                            
    class      = network                                                                                                                                                                                                                                                                                                     
    subclass   = ethernet                                                                                 

I test several times, and this issue can re-produce every time. If  i boot into windows and shutdown. no auto boot.

it seems the FreeBSD driver registered something wrong.
Comment 46 Kaho Toshikazu 2017-08-28 12:43:49 UTC
(In reply to Xuefeng deng from comment #45)

Thanks for your report. Is your system 11-stable?

Your symptom seems to be enabled WOL_UCAST or WOL_MCAST.
em(4) enables only WOL_MAGIC by default, if wol functionality is enabled
by the device EEPROM. If you use wol option for ifconfig, change it to wol_magic
or remove it.
Comment 47 Marius Strobl freebsd_committer freebsd_triage 2017-08-28 21:07:41 UTC
(In reply to Kaho Toshikazu from comment #46)

That's not quite correct, em(4) actually enables both WOL_MAGIC and WOL_MCAST
if APM wake is enabled according to EEPROM data:
        if (eeprom_data & apme_mask)
                adapter->wol = (E1000_WUFC_MAG | E1000_WUFC_MC);

I've no idea why it also enables WOL_MCAST in that case, though. That seems
like a bad choice which now hardly can be changed due to POLA reasons, i. e.
because a user may rely on WOL_MCAST also being on by default.
Comment 48 Kaho Toshikazu 2017-08-28 23:11:46 UTC
(In reply to Marius Strobl from comment #47)

if_em.c at r322955 in 11-stable:

5294	if (eeprom_data & apme_mask)
5295		adapter->wol = (E1000_WUFC_MAG | E1000_WUFC_MC);

Do you describe about it? Yes, it sets both WOL_MAGIC and WOL_MCAST.
But then only WOL_MAGIC is enabled in line 3260.
By default, em(4) sets all WOL capabilities and enables only WOL_MAGIC,
if EEPROM enabled WOL. If EEPROM disabled WOL, all WOL capabilities
are disabled.

3257	/* Enable only WOL MAGIC by default */
3258	if (adapter->wol) {
3259		if_setcapabilitiesbit(ifp, IFCAP_WOL, 0);
3260		if_setcapenablebit(ifp, IFCAP_WOL_MAGIC, 0);
3261	}

 At suspend or shutdown time, these bits are cleared,
if the functionalities are disabled.

5361	/*
5362	** Determine type of Wakeup: note that wol
5363	** is set with all bits on by default.
5364	*/
5365	if ((if_getcapenable(ifp) & IFCAP_WOL_MAGIC) == 0)
5366		adapter->wol &= ~E1000_WUFC_MAG;
5367
5368	if ((if_getcapenable(ifp) & IFCAP_WOL_MCAST) == 0)
5369		adapter->wol &= ~E1000_WUFC_MC;
Comment 49 Xuefeng deng 2017-08-29 01:36:48 UTC
(In reply to Kaho Toshikazu from comment #46)
Yes, my system is 11-table (svn up to 322963).
I will test it tonight with wol_magic.

and is there anyway to read the nic register value on windows? then we can do some comparision.
Comment 50 Xuefeng deng 2017-08-29 11:17:00 UTC
I did a test with wol_magic, it continue auto boot after shutdown.
Comment 51 Marius Strobl freebsd_committer freebsd_triage 2017-08-30 00:18:44 UTC
Created attachment 185903 [details]
MFC r322986 to stable/11
Comment 52 Marius Strobl freebsd_committer freebsd_triage 2017-08-30 00:27:58 UTC
(In reply to Xuefeng deng from comment #50)

Okay, please do the following things:
1) Show us the output of `ifconfig em0` after boot so we see what WOL options
   actually got enabled or not.

2) Issue `ifconfig em0 -wol` and then check whether the machine powers itself
   up again after a `shutdown -p`.

3) If the machine powers up itself again with 2), retest 2) using a kernel
   built with the following patch:
   https://bz-attachments.freebsd.org/attachment.cgi?id=185903
Comment 53 Kaho Toshikazu 2017-08-30 00:40:37 UTC
(In reply to Xuefeng deng from comment #50)

Thanks for your reports.

Please show both the "options=" and "capabilities=" lines
of `ifconfig -m em0` outputs.

If you have WOL_MCAST or WOL_UCAST in the "options=" line,
disable these with `ifconfig em0 -wol_ucast -wol_mcast wol_magic`.
 
If you have only WOL_MAGIC but your machine boots immediately,
please exam only WOL_UCAST options with `ifconfig em0 wol_ucast -wol_mcast -wol_magic`. This setting is not good for usual use, but a hint
may be revealed by it. 

If your machine boots automatically even if all WOL options were disabled,
something would be broken.

(In reply to Xuefeng deng from comment #49)

I'm sorry. I don't know about Windows technique, 
but don't worry about it because Intel publishes the datasheet of i219.
Comment 54 Kaho Toshikazu 2017-08-30 01:40:57 UTC
(In reply to Marius Strobl from comment #51)

Hmm, can you revert it?

I don't test it. I think that if EEPROM enabled WOL, this commit
would not make any differences, but if EEPROM disabled WOL and OS enabled WOL,
a machine would not boot by WOL.

In r321257, the parts of "restore WUC/WUFC update" also broke, but
the part of the codes which you changed by r322986 recovered this breakage,
I think. I'm afraid that some system can not use WOL now.

On WOL functionality, 12-current is different from 11-stable. 
12-current: always capable, can force to enable
11-stable:  capable if enabled, can not force to enable
I think this commit breaks "can force to enable" capability.
Comment 55 Xuefeng deng 2017-08-30 14:16:35 UTC
I think I found the problem:
1. I have two NICs, one is em0, the othere are igb[0-3]
2. when wol (while also include WOL_MCAST) is configured in rc.conf, no matter on em0 or on igb[0-3], system will auto started after "shutdown -p now"
3. my previous auto boot was caused by wol option on igb0
4. if only configured wol_magic, no auto boot issue.

so. WOL_MCAST (for both em0 or igb[0-3]) is the root cause which make system auto started. 

ifconfig -m em0:
==> capabilities=15399b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_UCAST,WOL_MCAST,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,NETMAP>
NOTE: in my host, wol_ucast does not in options after via ifconfig em0 wol_ucast.


before or after patch has no change:
1.1 remove wol in /etc/rc.conf
1.2 ifconfig em0 -wol
1.3 ifconfig em0 ==> options=4019b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,VLAN_HWTSO>
1.4 shutdown -p now
1.5 no auto boot
1.6 execute (wakeonlan MAC_ADDR) from another host. system not started

2.0 press power and boot into freebsb
2.1 ifconfig em0 ==> options=4219b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_MAGIC,VLAN_HWTSO> // Why wol_magic is configured? from EEPROM?
2.2 shutdown -p now
2.3 no auto booted
2.4 ifconfig em0 -wol
2.5 ifconfig em0 ==> options=4019b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,VLAN_HWTSO>
2.6 shutdown -r now
2.7 ifconfig em0 ==> options=4219b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_MAGIC,VLAN_HWTSO>

3.0 ifconfig em0 -wol
3.1 ifconfig em0 wol_mcast ==> options=4119b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_MCAST,VLAN_HWTSO>
3.2 shutdown -p now
3.3 auto booted
3.4 ifconfig em0 ==> options=4219b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_MAGIC,VLAN_HWTSO>
Comment 56 Kaho Toshikazu 2017-08-30 15:50:07 UTC
(In reply to Xuefeng deng from comment #55)

Thanks for your reports.

Your machine operates normally and em driver does not have the WOL problem.
Please don't enable WOL_UCAST and WOL_MCAST. Don't use wol options,
if you don't want a machine boot unexpectedly. 
Because a probability of a magic packet generated coincidentally is very low,
you can use WOL_MAGIC safely.

I had worried WUFC register had some garbage bit, but my thought was incorrect.
I think your problem happened like this.
Before MFC, for your device, em driver does not support WOL.
You miss-configured wol, but it was simply ignored.
After MFC, em driver supports WOL, and your setting made correct effect.

Even if you use other devices, please enable only WOL_MAGIC. Otherwise,
your machine may boot unexpectedly.
Comment 57 Xuefeng deng 2017-08-31 06:09:25 UTC
(In reply to Kaho Toshikazu from comment #56)
yes, only use wol_magic works well.

if I only set wol on igb[0-3] (no wol or wol_magic on em0), the system will also auto boot.

so the igb code should also has the same issue on my host.
Comment 58 Kaho Toshikazu 2017-08-31 07:53:17 UTC
(In reply to Xuefeng deng from comment #57)

> if I only set wol on igb[0-3] (no wol or wol_magic on em0), 
> the system will also auto boot.

  Do you have wol option on igb[0-3]? If you have, please remove it.
It is not intel nic specific problem but also other nic have.
If you would like to enable WOL, you should use wol_magic instead of wol.
If you would like to use more reliable option, 
use "-wol_ucast -wol_mcast wol_magic".

Before r322956, igb device cannot boot from WOL even if ifconfig command
showed a WOL functionality was enabled. Even if EEPROM enabled WOL,
FreeBSD igb(4) disables it and never enable.

After r322956, igb disconnects all WOL related events from WOL at initializing
and then forces to enable WOL and connects to some events at shutdown time.
After r322955, em almost does not do nothing related WOL at initializing, and
connects to some events at shutdown time. 
If EEPROM disabled WOL, em cannot use WOL but igb can enable WOL.
Comment 59 commit-hook freebsd_committer freebsd_triage 2017-09-01 00:00:30 UTC
A commit references this bug:

Author: marius
Date: Thu Aug 31 23:59:47 UTC 2017
New revision: 323080
URL: https://svnweb.freebsd.org/changeset/base/323080

Log:
  MFC: r308643, r312427, r312641, r322986

  - Update WOL support for newer em(4) devices. [1]
  - Add support for Kaby Lake generation i219 (4) and i219 (5) devices.
  - Enable WOL features also for the igb(4) class of devices. [1]
  - Don't set any WOL enabling hardware bits if WOL isn't requested
    according to the enabled interface capability bits.

  PR:		208343 [1]
  Submitted by:	Kaho Tashikazu <kaho@elam.kais.kyoto-u.ac.jp> [1]
  Approved by:	re (kib)

Changes:
_U  stable/10/
  stable/10/sys/dev/e1000/e1000_82575.c
  stable/10/sys/dev/e1000/e1000_82575.h
  stable/10/sys/dev/e1000/e1000_defines.h
  stable/10/sys/dev/e1000/e1000_ich8lan.c
  stable/10/sys/dev/e1000/if_em.c
  stable/10/sys/dev/e1000/if_igb.c
Comment 60 sandreas 2018-05-17 17:01:55 UTC
Fix does not work on my hardware (ASRock Rack C236 WSI)... see bug https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=228302