Bug 290210 - sys/netinet/multicast:IP_ADD_MEMBERSHIP_ip_mreq tests fails intermittently in CI
Summary: sys/netinet/multicast:IP_ADD_MEMBERSHIP_ip_mreq tests fails intermittently in CI
Status: Closed FIXED
Alias: None
Product: Base System
Classification: Unclassified
Component: tests (show other bugs)
Version: 16.0-CURRENT
Hardware: Any Any
: --- Affects Only Me
Assignee: Gleb Smirnoff
URL:
Keywords: regression
Depends on:
Blocks:
 
Reported: 2025-10-13 15:34 UTC by Siva Mahadevan
Modified: 2025-10-14 23:45 UTC (History)
2 users (show)

See Also:

Attachments
kyua test report (2.05 KB, text/plain)
2025-10-13 15:34 UTC, Siva Mahadevan 		no flags Details
suggested patch (2.60 KB, patch)
2025-10-14 18:19 UTC, Gleb Smirnoff 		no flags Details | Diff
View All Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.
Description Siva Mahadevan freebsd_committer freebsd_triage 2025-10-13 15:34:38 UTC 
Created attachment 264542 [details]
kyua test report

The 'sys/netinet/multicast:IP_ADD_MEMBERSHIP_ip_mreq' testcase fails intermittently in CI:


===> sys/netinet/multicast:IP_ADD_MEMBERSHIP_ip_mreq 
Result:     failed: atf-check failed; see the output of the test for details

Standard output: 
Executing command [ jexec mjail1 /usr/tests/sys/netinet/multicast-send 0.0.0.0 6676 233.252.0.1 6676 192.0.2.1 hello ] 
Executing command [ sh -c wait 15524; exit 0 ] 
Executing command [ cat out ] 
 
Standard error: 
Fail: stdout does not match expected value 
--- /tmp/kyua.JslG9E/2/work/atf-check.k1LunD    2025-10-13 10:31:06.916151000 +0000 
+++ /tmp/kyua.JslG9E/2/work/check.ZAAQYc/stdout 2025-10-13 10:31:06.915409000 +0000 
@@ -1 +0,0 @@ 
-192.0.2.1:6676 hello



I have reproduced this error using Bricoler with 100 runs for good measure and attached the failing test report in this bug:

$ bricoler run freebsd-src-regression-suite --param freebsd-src:url=/usr/src --param freebsd-src:branch= --param freebsd-src-regression-suite:hypervisor=bhyve --param freebsd-src-regression-suite:memory=4096 --param freebsd-src-regression-suite:ncpus=2 --param freebsd-src-regression-suite:parallelism=1 --param freebsd-src-regression-suite:count=100 --param freebsd-src-regression-suite:tests='sys/netinet/multicast:IP_ADD_MEMBERSHIP_ip_mreq'

@glebius, since you made the last significant edit to the tests in https://cgit.freebsd.org/src/commit/?id=d0c8a0b3e6fc5372428eb4fddcd4f8c65b5b0ee0 (tests/netinet: extend IPv4 multicast testing framework), please triage as necessary. Note that the sys/netinet/multicast:MCAST_JOIN_GROUP also fails intermittently.
Comment 1 Gleb Smirnoff freebsd_committer freebsd_triage 2025-10-14 18:19:22 UTC 
Created attachment 264580 [details]
suggested patch

Can you please do me a favor and test this patch in your environment?
Comment 2 Siva Mahadevan freebsd_committer freebsd_triage 2025-10-14 19:09:22 UTC 
Yes this seems to work. After the patch, these tests don't fail even with 200+ consecutive runs each.
Comment 3 commit-hook freebsd_committer freebsd_triage 2025-10-14 23:45:00 UTC 
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=c8359b327104325af6dc97202e21d6131acb0573

commit c8359b327104325af6dc97202e21d6131acb0573
Author:     Gleb Smirnoff <glebius@FreeBSD.org>
AuthorDate: 2025-10-14 23:43:15 +0000
Commit:     Gleb Smirnoff <glebius@FreeBSD.org>
CommitDate: 2025-10-14 23:43:15 +0000

    tests/multicast: fix test flakyness

    The receiver is run as a background job, so we need a wait loop to
    make sure it has been fully functional before launching sender.

    Reported by:    Siva Mahadevan <me svmhdvn.name>
    PR:             290210

 tests/sys/netinet/multicast.sh | 29 ++++++++++++++++-------------
 1 file changed, 16 insertions(+), 13 deletions(-)