In 12-CURRENT, I would like to be able to turn off all NIC offloading features (TSO,LRO,Checksums) to ensure I am reading everything as it appears off the wire for network security (Bro NSM). Putting all of the flags together, this is what I have as the line that should disable all of the features /sbin/ifconfig em0 -rxcsum -txcsum -tso4 -tso6 -lro -rxcsum6 -txcsum6 -vlanhwcsum -vlanhwtso It appears most work with and are enabled and disabled, the exceptions are rxcsum, rxcsum6 and vlanhwcsum. I cannot disable vlanhwcsum at all, and when you disable rxcsum, it enables rxcsum6 and vice versa. Example: # ifconfig em0 em0: flags=28943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST,PPROMISC> metric 0 mtu 1500 options=810099<RXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,VLAN_HWFILTER> ether 68:05:ca:xx:xx:xx media: Ethernet autoselect (1000baseT <full-duplex>) status: active nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL> # ifconfig em0 -rxcsum # ifconfig em0 em0: flags=28943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST,PPROMISC> metric 0 mtu 1500 options=a10098<VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,VLAN_HWFILTER,RXCSUM_IPV6> ether 68:05:ca:xx:xx:xx media: Ethernet autoselect (1000baseT <full-duplex>) status: active nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL> # ifconfig em0 -rxcsum6 # ifconfig em0 em0: flags=28943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST,PPROMISC> metric 0 mtu 1500 options=810099<RXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,VLAN_HWFILTER> ther 68:05:ca:xx:xx:xx media: Ethernet autoselect (1000baseT <full-duplex>) status: active nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL> I see the same behavior with this driver on two separate cards. em0@pci0:1:0:0: class=0x020000 card=0x115e8086 chip=0x105e8086 rev=0x06 hdr=0x00 vendor = 'Intel Corporation' device = '82571EB/82571GB Gigabit Ethernet Controller D0/D1 (copper applications)' class = network subclass = ethernet em0@pci0:0:31:6: class=0x020000 card=0x06d91028 chip=0x15b78086 rev=0x31 hdr=0x00 vendor = 'Intel Corporation' device = 'Ethernet Connection (2) I219-LM' class = network subclass = ethernet If this is "by design" in the driver, then this bug can be closed, but I would assume that if you want to disable all receive side offloading, you should be able to disable it for both IPv4 and IPv6. Also, if this is a driver issue (iflib), then I can move the bug to kernel component.
Created attachment 197023 [details] vlan_hwcsum flag sets changeable. (In reply to Shirkdog from comment #0) I think it is only a iflib bug. See bug #220698 . Please try the attached patch.
Yeah, for the RX checksum stuff, it's pretty clear that if *either* rxcsum or rxcsum6 is set in the target capabilities, both will be forced on. Since ifconfig does a separate ioctl for each command-line flag, there's never a set that has both set to zero, so neither ever gets disabled. I'll see if I can figure out why r307562 made this change, and come up with a Better Way to twi the two flags together.
(In reply to Stephen Hurd from comment #2) > I'll see if I can figure out why r307562 made this change, and come up > with a Better Way to twi the two flags together. I don't know exactly but igb devices are not able to enable/disable rxcsum and rxcsum6 separately by the hardware level limitation. If only one rxcsum or rxcsum6 flag is changed, another flag would like to be changed forcefully to hold same state. I think iflib should not do a special thing, and it is better that an individual driver has some workarounds if need.
(In reply to Kaho Toshikazu from comment #3) I think that is it okay for this, if someone wants to disable a NIC feature for IPv4, there is no reason to leave it enabled for IPv6. I am also rebuilding a kernel to test the patch attached this bug.
Created attachment 197054 [details] Allow rxcsum to be disabled, but keep it tied to rxcsum6 This patch allows rxcsum to be disabled, but retains the restriction that both flags be toggled together. Still looking into the VLAN_HWCSUM issue.
(In reply to Kaho Toshikazu from comment #3) > I think iflib should not do a special thing, and it is better that > an individual driver has some workarounds if need. The driver isn't involved in this ioctl, so it needs to support the lowest common denominator of iflib devices. This ioctl works by stopping the driver, setting the capabilities, then restarting the driver... which then applies the capabilities. If a driver can't support a particular combination of flags, there's no way for it to tell iflib at this time. For rxcsum, the ifconfig specifically mentions that rxcsum and rxcsum6 may not be interdependent. It looks like there's some brokenness in the WoL stuff too. I'm digging into that once I verify your vlan_hwcsum changes.
(In reply to Stephen Hurd from comment #5) > + mask &= ctx->ifc_softc_ctx.isc_capabilities; An em class device doesn't have rxcsum6 capability and this line is useful. If a device doesn't have rxcsum6 capability, don't touch a flag related rxcsum6 and the variable setmask should be also masked by isc_capabilities, I think. > It looks like there's some brokenness in the WoL stuff too. No, it is not broken. But the Wol feature doesn't have any effect until shutdown or suspend time, and set flags only and avoid re-initialization. > I'm digging into that once I verify your vlan_hwcsum changes. The vlan_hwcsum is simply forgot in the definition IFCAP_FLAGS. If the IFCAP_FLAGS has the Wol flags, any special masking operation is not need.
(In reply to Kaho Toshikazu from comment #7) > the variable setmask should be also masked by isc_capabilities, I think. Everything going into setmask should be masked by mask already... I've got that fixed in my patch now, will update right away. > No, it is not broken. I meant in the existing code, not in your patch. I'm trying to figure out why the old code was forcing only IFCAP_WOL_MCAST|IFCAP_WOL_MAGIC if any WoL flag (including IFCAL_WOL_UCAST) was set. If a driver requires something like this, and the isc_capabilities isn't enough to restrict it, there needs to be a flag, a callback, or at least an update to some manpages regarding it. It's almost certainly better to set the WoL capabilities as specified and have some not work then it is to not allow some to be specified at all or to change what set is enabled. > But the Wol feature doesn't have any effect until shutdown or suspend time, and set flags only and avoid re-initialization. Yeah, that part is good. We may actually want a larger set of things that don't reinit. > The vlan_hwcsum is simply forgot in the definition IFCAP_FLAGS. You also moved the if_vlancap(ifp) line to after the reinit. I'm trying to figure out if it should be set while the interface is stopped. It currently doesn't really make a difference as far as I can tell, but it may make sense to update while the interface is down, but then the context will be locked.
Created attachment 197061 [details] Update to include WoL and vlan hwcsum fixes
(In reply to Stephen Hurd from comment #8) > I'm trying to figure out why the old code was forcing only > IFCAP_WOL_MCAST|IFCAP_WOL_MAGIC if any WoL flag (including IFCAL_WOL_UCAST) > was set. I suspect that because of a machine is easily woken up by a unintended packet if wol_ucast is enabled. > You also moved the if_vlancap(ifp) line to after the reinit. A vlan interface requires the flags of the parent interface. Because of very limited flags are passed through, in the existing code, the vlan interface can usually get correct flags but sometimes get old incorrect flags.
(In reply to Stephen Hurd from comment #9) rebuilt and tested using the update and they appear to be honored (running ifconfig em0 promisc up at boot: # ifconfig em0 em0: flags=28943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST,PPROMISC> metric 0 mtu 1500 options=81049b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,LRO,VLAN_HWFILTER> ether 68:05:ca:33:42:f0 media: Ethernet autoselect (1000baseT <full-duplex>) status: active nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL> # ifconfig em0 -rxcsum -txcsum -tso4 -tso6 -lro -rxcsum6 -txcsum6 -vlanhwcsum -vlanhwtso # ifconfig em0 em0: flags=28943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST,PPROMISC> metric 0 mtu 1500 options=810018<VLAN_MTU,VLAN_HWTAGGING,VLAN_HWFILTER> ether 68:05:ca:33:42:f0 media: Ethernet autoselect (1000baseT <full-duplex>) status: active nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL> I did notice that I no longer observed the any of the IPv6 flags when I try to enable them, and this interface has no IP configuration. But I am testing sniffing on the interface and will report back if I have any issues.
(In reply to Shirkdog from comment #11) > I did notice that I no longer observed the any of the IPv6 flags > when I try to enable them, It is correct because em doesn't have ipv6 offloads. You can see the capabilities of the device with `ifconfig -m em0`. The old code changes the rxcsum6 flag in spite of its absence capability.
A commit references this bug: Author: shurd Date: Thu Sep 20 19:35:35 UTC 2018 New revision: 338838 URL: https://svnweb.freebsd.org/changeset/base/338838 Log: Fix capabilities handling for iflib drivers Various capabilities were not being handled correctly in the SIOCSIFCAP handler. Specifically: IFCAP_RXCSUM and IFCAP_RXCSUM_IPV6 could be set even if not supported It was impossible to disable IFCAP_RXCSUM and/or IFCAP_RXCSUM_IPV6 via ifconfig since it does ioctl() per command-line flag rather than combine them into a single call. IFCAP_VLAN_HWCSUM could not be modified via the ioctl() Setting any combination of the three IFCAP_WOL flags would set only IFCAP_WOL_MCAST | IFCAP_WOL_MAGIC. For example, setting only IFCAP_WOL_UCAST would result in both IFCAP_WOL_MCAST and IFCAP_WOL_MAGIC being enabled, but IFCAP_WOL_UCAST would not be enabled. Because if_vlancap() was called before if_togglecapenable(), vlan flags were sometimes not applied correctly. Interfaces were being unnecessarily stopped and restarted for WoL PR: 231151 Submitted by: Kaho Toshikazu <kaho@elam.kais.kyoto-u.ac.jp> Reported by: Shirkdog <mshirk@daemon-security.com> Reviewed by: galladin Approved by: re (gjb) Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D17158 Changes: head/sys/net/iflib.c