The following tests are failing on amd64/i386 on ^/head and recent back ports to ^/stable/12: sys.netpfil.pf.forward.v4 sys.netpfil.pf.forward.v6 sys.netpfil.pf.fragmentation.v6 sys.netpfil.pf.icmp.cve_2019_5598 sys.netpfil.pf.set_tos.v4 The error message seems to be coming from scapy. From https://ci.freebsd.org/job/FreeBSD-head-amd64-test/10846/testReport/junit/sys.netpfil.pf/forward/v4/ : ==================================== Fail: incorrect exit status: 1, expected: 0 stdout: stderr: Traceback (most recent call last): File "/usr/tests/sys/netpfil/pf/pft_ping.py", line 4, in <module> import scapy.all as sp File "/usr/local/lib/python2.7/site-packages/scapy/all.py", line 27, in <module> from scapy.route import * File "/usr/local/lib/python2.7/site-packages/scapy/route.py", line 203, in <module> conf.iface = get_working_if() File "/usr/local/lib/python2.7/site-packages/scapy/arch/bpf/core.py", line 180, in get_working_if ifaces = get_working_ifaces() File "/usr/local/lib/python2.7/site-packages/scapy/arch/bpf/core.py", line 164, in get_working_ifaces interfaces.append((ifname, int(ifname[-1]))) ValueError: invalid literal for int() with base r10: 'a' Files left in work directory after failure: created_interfaces.lst, created_jails.lst ==================================== All in all, this looks like package breakage, but it's resulting in failed tests.
This isn't happening on my ^/head@r346039 system, so it might be something related to the CI environment, or something specific to a later revision.
Recently I have updated scapy plus some of it's dependency like libdnet. Unfortunately the previous libdnet actually couldn't detect pf/bpf properly and would not compile the relevent codes applicable to a FreeBSD system. So I have changed some detection part and now it compiles with pf/bpf support. I will need to check in details whether if this is a pf issue or scapy issue. Can anyone please share the pipelines for our builds? Or is it publicly available? I can't login to https://ci.freebsd.org with my freefall creds.
(In reply to Muhammad Moinur Rahman from comment #2) All the codes are available here: https://github.com/freebsd/freebsd-ci I think you can test by just downloading the test disk image from: https://artifact.ci.freebsd.org/snapshot/head/latest_vm/amd64/amd64/disk-test.img.xz And install the new scapy to test.
I think this might help: https://github.com/secdev/scapy/pull/1990/
Created attachment 203760 [details] Patch from https://github.com/secdev/scapy/pull/1991
A commit references this bug: Author: lwhsu Date: Thu Apr 18 18:50:57 UTC 2019 New revision: 499307 URL: https://svnweb.freebsd.org/changeset/ports/499307 Log: Add a patch from upstream which solves the interface name may not end by digit PR: 237305 Obtained from: https://github.com/secdev/scapy/pull/1991 Sponsored by: The FreeBSD Foundation Changes: head/net/scapy/Makefile head/net/scapy/files/1991.patch
New package is available on the official mirror and test are passing in the CI system.