These 3 tests always fail on i386: sys.netpfil.pf.forward.v4 sys.netpfil.pf.forward.v6 sys.netpfil.pf.set_tos.v4 There are some analysis from kp@: https://lists.freebsd.org/pipermail/freebsd-testing/2019-June/001933.html https://lists.freebsd.org/pipermail/freebsd-testing/2019-June/001934.html
A commit references this bug: Author: lwhsu Date: Mon Jul 22 18:54:26 UTC 2019 New revision: 350221 URL: https://svnweb.freebsd.org/changeset/base/350221 Log: Temporarily skip sys.netpfil.pf.forward.{v4,v6} and sys.netpfil.pf.set_tos.v4 on i386 as they are flakey on it PR: 239380 Sponsored by: The FreeBSD Foundation Changes: head/tests/sys/netpfil/pf/forward.sh head/tests/sys/netpfil/pf/set_tos.sh
A commit references this bug: Author: lwhsu Date: Tue Jul 23 08:20:53 UTC 2019 New revision: 350237 URL: https://svnweb.freebsd.org/changeset/base/350237 Log: MFC r350221 Temporarily skip sys.netpfil.pf.forward.{v4,v6} and sys.netpfil.pf.set_tos.v4 on i386 as they are flakey on it PR: 239380 Sponsored by: The FreeBSD Foundation Changes: _U stable/12/ stable/12/tests/sys/netpfil/pf/forward.sh stable/12/tests/sys/netpfil/pf/set_tos.sh
(In reply to Li-Wen Hsu from comment #0) kp suggests it appears to be a bpf issue
It looks like I was wrong and this isn't actually a problem in bpf, but with scapy itself. With scapy-2.3.3 the tests pass, with py27-scapy-2.4.2_2 they do not.
A commit references this bug: Author: bofh Date: Thu Sep 5 13:26:15 UTC 2019 New revision: 511187 URL: https://svnweb.freebsd.org/changeset/ports/511187 Log: net/scapy: Update version 2.4.2=>2.4.3 - Use net/py-pypcap instead of net/py-pcap as dnet and py-pcap will be deprecated in future scapy versions. One other reason is py-pcap is failing with python3. [1] PR: 239380 [1] Submitted by: lwhsu [1] Changes: head/net/scapy/Makefile head/net/scapy/distinfo head/net/scapy/files/
A commit references this bug: Author: bofh Date: Wed Sep 11 10:24:29 UTC 2019 New revision: 511811 URL: https://svnweb.freebsd.org/changeset/ports/511811 Log: inet/scapy: Update version 2.4.2=>2.4.3 - Use net/py-pypcap instead of net/py-pcap as dnet and py-pcap will be deprecated in future scapy versions. One other reason is py-pcap is failing with python3. [1] - Change MASTER_SITE to PYPI for better availability [2] - Update WWW [2] PR: 239380 [1] 240493 [2] Submitted by: lwhsu [1] [2] Changes: head/net/scapy/Makefile head/net/scapy/distinfo head/net/scapy/files/ head/net/scapy/pkg-descr
scapy 2.4.3 installed: https://ci.freebsd.org/job/FreeBSD-head-i386-testvm/12053/console but the test job: https://ci.freebsd.org/job/FreeBSD-head-i386-test/6717/ still has these two cases failing: https://ci.freebsd.org/job/FreeBSD-head-i386-test/6717/testReport/sys.netpfil.common/forward/pf_v4/ https://ci.freebsd.org/job/FreeBSD-head-i386-test/6717/testReport/sys.netpfil.common/tos/pf_tos/
(In reply to Li-Wen Hsu from comment #7) Is it possible to comment out the RUN_DEPENDS and rerun the test without committing ?
(In reply to Muhammad Moinur Rahman from comment #8) It's possible by manually building the package and installing to the VM. Do you mean all the RUN_DEPENDS? Can you provide a patch?
I tried to remove py27-pypcap-1.2.3, libdnet-1.13_3, py27-libdnet-1.13_3 with `pkg -f` to keep py27-scapy-2.4.3, but the tests are still failing.
Just tried with py36-scapy-2.4.3_1 again, with and without py36-pypcap-1.2.3 and py36-libdnet-1.13_3, both failed.
A commit references this bug: Author: bz Date: Tue Nov 5 10:53:56 UTC 2019 New revision: 354357 URL: https://svnweb.freebsd.org/changeset/base/354357 Log: frag6 tests: set end to catch timeout as error There are times when we have to wait for reply packets. There are either an ICMPv6 (error) reply or the expiration timeout. In these cases synchonous ICMPv6 replies should arrive, always, unless the packet is lost. Due to errors experienced with the test software sending an invlaid request on at least i386 (*) these packets are not generated. That means we are waiting for a long time for the replies or even timeout the test case. Manually set the "End" flag on these test cases as well, so they do fail rather than timeout as the sniffer timeout happens. This improves debugging options, reflects the error properly, and saves time on each test suit run. (*) The real cause for that is still to be found (see the referenced PRs) PR: 241493, 239380 MFC after: 2 weeks Sponsored by: Netflix Changes: head/tests/sys/netinet6/frag6/frag6_02.py head/tests/sys/netinet6/frag6/frag6_04.py
*** Bug 241493 has been marked as a duplicate of this bug. ***
*** Bug 240085 has been marked as a duplicate of this bug. ***
*** Bug 240086 has been marked as a duplicate of this bug. ***
*** Bug 241739 has been marked as a duplicate of this bug. ***
Created attachment 208911 [details] Fixes for scapy bpf on FreeBSD (i386) Hi, this diff is from the installed version of scapy and not relative to the port. It fixes both the alignment on (32bit) platforms where sizeof(long) == 4 and for i386 the offset on the bpf_hdr struct as time_t still is 32bit. Scapy does not set any more specific BPF_T_* time option, so we are not switching to bpf_xhdr (which would probably be the better long-term solution). That said, the current situation in FreeBSD bpf is a mess until we BURN_BRIDGES on the old bpf_hdr. I have not yet submitted a ports PR or the change upstream to scapy. This work was Sponsored by: Netflix
(In reply to Bjoern A. Zeeb from comment #17) As a follow-up. The problem for the test cases was as follows (at least for the IPv6 ones). To resolve the neighbour's address, scapy sends a NS. The kernel of the destination/next hop replies with a NA. Scapy receives the packets but due to the wrong offsets within the bpf header the frame sizes are all way off and way too large. As a result there is no result packet to the internal AsyncSniffer and neither the original packet nor the NA reply is seen. With the failed address resolution scapy uses a broadcast Ethernet destination MAC address on the ethernet packets. FreeBSD will mark this packet with M_BCAST as a result and higher up in the kernel certain functions will check for that and get an invalid result (or not accept the packet, or not trigger and ICMPv6 reply). With that the expected reply packet from the test cases are missing and the sniffer there will not see any further packets and either timeout or checks will fail and with that the test times out or fails. Hope that explains some of this.
A commit references this bug: Author: bz Date: Wed Nov 6 14:43:36 UTC 2019 New revision: 354395 URL: https://svnweb.freebsd.org/changeset/base/354395 Log: IPv6 tests: test case for scapy bpf parsing bug PR: 239380 Reported by: lwhsu, CI system MFC after: 3 weeks Sponsored by: Netflix Changes: head/tests/sys/netinet6/Makefile head/tests/sys/netinet6/scapyi386.py head/tests/sys/netinet6/scapyi386.sh
A commit references this bug: Author: lwhsu Date: Wed Nov 6 17:56:48 UTC 2019 New revision: 516910 URL: https://svnweb.freebsd.org/changeset/ports/516910 Log: Add a patch which fixes both the alignment on (32bit) platforms where sizeof(long) == 4 and for i386 the offset on the bpf_hdr struct as time_t still is 32bit. PR: 239380 Approved by: bofh (maintainer) Sponsored by: Netflix (bz) Sponsored by: The FreeBSD Foundation (lwhsu) Changes: head/net/scapy/Makefile head/net/scapy/files/ head/net/scapy/files/patch-i386
Submitted upstream: https://github.com/secdev/scapy/pull/2329
A commit references this bug: Author: lwhsu Date: Fri Nov 8 18:56:04 UTC 2019 New revision: 354553 URL: https://svnweb.freebsd.org/changeset/base/354553 Log: Reenable netinet6 and netpfil tests on i386, net/scapy 2.4.3_2 contains the fix PR: 239380 Sponsored by: The FreeBSD Foundation Changes: head/tests/sys/netinet6/exthdr.sh head/tests/sys/netinet6/scapyi386.sh head/tests/sys/netpfil/common/forward.sh head/tests/sys/netpfil/common/tos.sh head/tests/sys/netpfil/pf/forward.sh head/tests/sys/netpfil/pf/set_tos.sh
bz@ has fixed this problem, and lwhsu@ has re-activated the tests. This bug can be closed.
A commit references this bug: Author: bz Date: Wed Jan 8 16:32:18 UTC 2020 New revision: 356493 URL: https://svnweb.freebsd.org/changeset/base/356493 Log: MFC r354357-354358,354390,354395: frag6 tests: set end to catch timeout as error ipv6 tests: Add very simplistic extension header tests ipv6 tests: cleanup ipv6 tests: test case for scapy bpf parsing bug PR: 239380 Changes: _U stable/12/ stable/12/tests/sys/netinet6/Makefile stable/12/tests/sys/netinet6/exthdr.py stable/12/tests/sys/netinet6/exthdr.sh stable/12/tests/sys/netinet6/frag6/frag6.subr stable/12/tests/sys/netinet6/frag6/frag6_02.py stable/12/tests/sys/netinet6/frag6/frag6_04.py stable/12/tests/sys/netinet6/scapyi386.py stable/12/tests/sys/netinet6/scapyi386.sh