Bug 289821 - sys/netpfil/pf/mld.py:TestMLD::test_router_alert test fails intermittently in CI
Summary: sys/netpfil/pf/mld.py:TestMLD::test_router_alert test fails intermittently in CI
Status: Closed FIXED
Alias: None
Product: Base System
Classification: Unclassified
Component: tests (show other bugs)
Version: 16.0-CURRENT
Hardware: Any Any
: --- Affects Only Me
Assignee: Kristof Provost
URL:
Keywords: regression
Depends on:
Blocks:
 
Reported: 2025-09-24 17:53 UTC by Siva Mahadevan
Modified: 2025-10-12 03:02 UTC (History)
1 user (show)

See Also:


Attachments
kyua test report (7.65 KB, text/plain)
2025-09-24 17:53 UTC, Siva Mahadevan
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Siva Mahadevan freebsd_committer freebsd_triage 2025-09-24 17:53:29 UTC
Created attachment 264067 [details]
kyua test report

The 'sys/netpfil/pf/mld.py:TestMLD::test_router_alert' testcase fails intermittently in CI:

===> Failed tests
sys/netpfil/pf/mld.py:TestMLD::test_router_alert  ->  failed: /usr/tests/sys/netpfil/pf/mld.py:84: AssertionError  [6.084s]


I have reproduced this error using Bricoler with 100 runs for good measure and attached the failing test report in this bug:

$ bricoler run freebsd-src-regression-suite --param freebsd-src:url=/usr/src --param freebsd-src:branch= --param freebsd-src-regression-suite:hypervisor=bhyve --param freebsd-src-regression-suite:memory=4096 --param freebsd-src-regression-suite:ncpus=2 --param freebsd-src-regression-suite:parallelism=1 --param freebsd-src-regression-suite:count=100 --param freebsd-src-regression-suite:tests='sys/netpfil/pf/mld.py:TestMLD::test_router_alert'

@kp, since you added the tests in https://cgit.freebsd.org/src/commit/?id=b6d0caa485072b9b167a48959e37bd1871d80c6c (pf tests: verify we now allow MLD packets with the Router Alert extention header), please triage as necessary.
Comment 1 commit-hook freebsd_committer freebsd_triage 2025-09-30 12:54:08 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=a57f6ce4792f12bebdbe3e46d6d9d39da10055db

commit a57f6ce4792f12bebdbe3e46d6d9d39da10055db
Author:     Kristof Provost <kp@FreeBSD.org>
AuthorDate: 2025-09-27 14:41:30 +0000
Commit:     Kristof Provost <kp@FreeBSD.org>
CommitDate: 2025-09-30 12:53:21 +0000

    pf tests: fix intermittent mld test failures

    We can't reliably check for the absence of replies to our MLD queries (because
    a host may announce its multicast subscriptions), so enable pf logging and check
    for the relevant error message instead.

    PR:             289821
    MFC after:      1 week
    Sponsored by:   Rubicon Communications, LLC ("Netgate")
    Differential Revision:  https://reviews.freebsd.org/D52762

 tests/sys/netpfil/pf/Makefile |  2 +-
 tests/sys/netpfil/pf/mld.py   | 35 +++++++++++++++--------------------
 2 files changed, 16 insertions(+), 21 deletions(-)
Comment 2 commit-hook freebsd_committer freebsd_triage 2025-10-05 20:52:02 UTC
A commit in branch stable/15 references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=3e79ec993dc7f6a454705ff5e820aa977583a5f7

commit 3e79ec993dc7f6a454705ff5e820aa977583a5f7
Author:     Kristof Provost <kp@FreeBSD.org>
AuthorDate: 2025-09-27 14:41:30 +0000
Commit:     Kristof Provost <kp@FreeBSD.org>
CommitDate: 2025-10-05 20:47:35 +0000

    pf tests: fix intermittent mld test failures

    We can't reliably check for the absence of replies to our MLD queries (because
    a host may announce its multicast subscriptions), so enable pf logging and check
    for the relevant error message instead.

    PR:             289821
    MFC after:      1 week
    Sponsored by:   Rubicon Communications, LLC ("Netgate")
    Differential Revision:  https://reviews.freebsd.org/D52762

    (cherry picked from commit a57f6ce4792f12bebdbe3e46d6d9d39da10055db)

 tests/sys/netpfil/pf/Makefile |  2 +-
 tests/sys/netpfil/pf/mld.py   | 35 +++++++++++++++--------------------
 2 files changed, 16 insertions(+), 21 deletions(-)