The following testcases have been failing in CI since they were introduced: ``` lib.libc.net.getaddrinfo.getaddrinfo.basic lib.libc.net.getaddrinfo.getaddrinfo.nofamily lib.libc.net.getaddrinfo.getaddrinfo_test.basic lib.libc.net.getaddrinfo.getaddrinfo_test.empty_servname lib.libc.net.getaddrinfo.getaddrinfo_test.sock_raw ``` These tests will be disabled and need to be fixed--either in the libc code or in the tests themselves. More info can be found here: https://ci.freebsd.org/job/FreeBSD-main-amd64-test/26164/
I will use the new kyua allow_network_access introduced in ea1e4a7dd9bc to condition those tests (and other) if allow_network_access is set.
(In reply to Olivier Cochard from comment #1) The support has not been added yet. Once CI support is added, we should start relying on the variable -- otherwise the tests will always be skipped.
(In reply to Enji Cooper from comment #2) Ok, so there is a different problem I didn’t understand somewhere: If those new getaddrinfo tests are failing, it means the CI VM doesn’t have name resolution working (or Internet access). Which mean: We need to have those test disabled by default on the CI. Now, how the lib.libc.resolv/resolv_test and lib/libc/tests/nss/gethostby_test are able to return "pass" if the VM doesn’t have name resolution working ? Example: https://ci.freebsd.org/job/FreeBSD-main-amd64-test/26164/testReport/junit/lib.libc.resolv/resolv_test/ $ echo "nameserver 127.0.0.1" | sudo tee /etc/resolv.conf $ kyua test lib/libc/nss/gethostby_test:gethostbyname2_getaddrinfo_ipv4 lib/libc/nss/gethostby_test:gethostbyname2_getaddrinfo_ipv4 -> passed [0.010s] Results file id is usr_tests.20250401-203836-512466 Results saved to /home/olivier/.kyua/store/results.usr_tests.20250401-203836-512466.db 1/1 passed (0 broken, 0 failed, 0 skipped) How they can’t be in "pass" state wihout being able to resolve name ??? $ truss lib/libc/nss/gethostby_test gethostbyname2_getaddrinfo_ipv4 using getaddrinfo() to resolve zeppo.rediris.es write(1,"using getaddrinfo() to resolve z"...,48) = 48 (0x30) open("/etc/hosts",O_RDONLY|O_CLOEXEC,0666) = 3 (0x3) fstat(3,{ mode=-rw-r--r-- ,inode=2562183,size=1189,blksize=4096 }) = 0 (0x0) read(3,"#\n# Host Database\n#\n# This fi"...,4096) = 1189 (0x4a5) read(3,0x4814b802b000,4096) = 0 (0x0) close(3) = 0 (0x0) socket(PF_INET,SOCK_DGRAM|SOCK_CLOEXEC,0) = 3 (0x3) connect(3,{ AF_INET 127.0.0.1:53 },16) = 0 (0x0) sendto(3,"\M-"4\^A\0\0\^A\0\0\0\0\0\0\^Eze"...,34,0,NULL,0) = 34 (0x22) poll({ 3/POLLRDNORM },1,5000) = 1 (0x1) recvfrom(3,0x4814b805a880,65536,0,0x199d2413d560,0x199d2413d14c) ERR#61 'Connection refused' close(3) = 0 (0x0) socket(PF_INET,SOCK_DGRAM|SOCK_CLOEXEC,0) = 3 (0x3) connect(3,{ AF_INET 127.0.0.1:53 },16) = 0 (0x0) sendto(3,"\M-"4\^A\0\0\^A\0\0\0\0\0\0\^Eze"...,34,0,NULL,0) = 34 (0x22) poll({ 3/POLLRDNORM },1,10000) = 1 (0x1) recvfrom(3,0x4814b805a880,65536,0,0x199d2413d560,0x199d2413d14c) ERR#61 'Connection refused' close(3) = 0 (0x0) passed
I've put my thoughts in https://reviews.freebsd.org/D49594#1131294 but I believe it's better to also to put here for the history (archive) reason. As phabricator might be gone in the future but the log of issue tracker should be preserved along with the project (as the commit log) @igoro wrote: ``` While ideally tests should not depend on external things as much as possible to have more reproducible nature, some of them may require to reach external resources, let's say to fetch something or to talk to the world services like DNS, etc. ``` Indeed and that's also why I am still hesitated to enable internet access on the jails in builder (when doing builds with the patches come from external) and when running the tests in VM. It's not only the security reason but also the matter of tests stability and reliability. In the case of DNS resolving, I suggest we just using the built-in local unbound for a simple local resolver to serve the test needs.
Patch here: https://reviews.freebsd.org/D49633
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=c10fd9ac00b1097ab622d055dcc3d1484012bccc commit c10fd9ac00b1097ab622d055dcc3d1484012bccc Author: Olivier Cochard <olivier@FreeBSD.org> AuthorDate: 2025-04-02 14:25:40 +0000 Commit: Olivier Cochard <olivier@FreeBSD.org> CommitDate: 2025-04-02 14:31:18 +0000 tests: Require allow_network_access for tests needing name resolution Tests that require working name resolution or network access now mandate that the kuya variable allow_network_access be set. PR: 285826 Reported by: ngie Reviewed by: igoro Approved by: lwhsu Sponsored by: Netflix Differential Revision: https://reviews.freebsd.org/D49633 lib/libc/tests/net/getaddrinfo/getaddrinfo.c | 13 +++++++++++-- 1 file changed, 11 insertions(+), 2 deletions(-)
Thanks all for your help and review.
Oops, this PR was referring to all the regressions, not just the 2 requesting network access, so I'm reopening it. Therefore, these three still need to be fixed: - lib.libc.net.getaddrinfo.getaddrinfo_test.basic - lib.libc.net.getaddrinfo.getaddrinfo_test.empty_servname - lib.libc.net.getaddrinfo.getaddrinfo_test.sock_raw
The failure of the other tests seems related to the non-default ip6addrctl_policy="ipv4_prefer" setting enabled on the test VM. This could be a remnant of a previous test or part of the VM configuration.