Bug 274850 - Packets are disappearing when both PF "divert-to" and "Dnpipe" rules are activated simultaneously
Summary: Packets are disappearing when both PF "divert-to" and "Dnpipe" rules are acti...
Status: Closed FIXED
Alias: None
Product: Base System
Classification: Unclassified
Component: kern (show other bugs)
Version: 14.0-RELEASE
Hardware: amd64 Any
: --- Affects Only Me
Assignee: Kristof Provost
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2023-11-01 14:07 UTC by Alfa
Modified: 2024-01-21 21:56 UTC (History)
2 users (show)

See Also:
kp: mfc-stable13-


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Alfa 2023-11-01 14:07:24 UTC
I am in the process of transitioning from IPFW's 'divert-to' to PF's 'divert-to.' Initially, I encountered a 'divert-to' loop problem, bug #272770, which has recently been resolved by @igor.ostapenko and @kp. Thanks for your work.
I manually applied your 'fix pf divert-to' loop patch to FreeBSD 14.0 RC2, as shown below:

https://reviews.freebsd.org/rGfabf705f4b5aff2fa2dc997c2d0afd62a6927e68
https://reviews.freebsd.org/rGc1146e6ad67fb866c2472a1cbe5609fd939fd5ef

When I loaded only the 'divert' rules shown below, everything worked as expected, and traffic flowed smoothly without any issues:

# divert rules
pass in log quick proto udp from any to port { 53 } divert-to 127.0.0.1 port 1234
pass in log quick proto tcp from any to port { 80 443 } divert-to 127.0.0.1 port 1234

However, when I added the 'dnpipe' rules below and reloaded the pf.conf using 'pfctl -e -f /etc/pf.conf,' all traffic related to ports r53, 80, and 443 disappeared. Subsequently, when I removed 'dnpipe 1001' and 'dnpipe 1' from the 'dnpipe' rules and reloaded the pf.conf, traffic began to flow as expected.

Thanks for your assistance in this matter.

# dnpipe limiter rules
ether pass in quick from ac:bc:aa:9c:32:09 l3 all tag captiveportal_auth_igb3 dnpipe 1001
ether pass out quick to ac:bc:aa:9c:32:09 l3 all tag captiveportal_auth_igb3 dnpipe 1
Comment 1 Igor Ostapenko 2023-11-13 18:19:19 UTC
(In reply to Alfa from comment #0)

Could you please provide a bit more details like what happens with diverted packets and what dummynet configuration is used for the pipes? If it's not behind some NDA.
Comment 2 Igor Ostapenko 2023-11-15 18:38:43 UTC
It’s on the early stage of the patch review, but probably you want to give it a try: https://reviews.freebsd.org/D42609?download=true. It could be helpful to know results of testing in the fields.
Comment 3 Alfa 2023-11-16 14:58:36 UTC
(In reply to Igor Ostapenko from comment #2)
Hi,

It worked

Thans for your work
Comment 4 commit-hook freebsd_committer freebsd_triage 2023-11-17 17:07:30 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=fe3bb40b9e807d4010617de1ef040ba3aa623487

commit fe3bb40b9e807d4010617de1ef040ba3aa623487
Author:     Igor Ostapenko <pm@igoro.pro>
AuthorDate: 2023-11-17 16:04:01 +0000
Commit:     Kristof Provost <kp@FreeBSD.org>
CommitDate: 2023-11-17 16:06:16 +0000

    pf: fix dummynet + ipdivert use case

    Dummynet re-injects an mbuf with MTAG_IPFW_RULE added, and the same mtag
    is used by divert(4) as parameters for packet diversion.

    If according to pf rule set a packet should go through dummynet first
    and through ipdivert after then mentioned mtag must be removed after
    dummynet not to make ipdivert think that this is its input parameters.

    At the very beginning ipfw consumes this mtag what means the same
    behavior with tag clearing after dummynet.

    And after fabf705f4b5a pf passes parameters to ipdivert using its
    personal MTAG_PF_DIVERT mtag.

    PR:             274850
    Reviewed by:    kp
    Differential Revision:  https://reviews.freebsd.org/D42609

 sys/netpfil/pf/pf.c               |  27 +++++++--
 tests/sys/netpfil/pf/divert-to.sh | 118 +++++++++++++++++++++++++++++++++++++-
 2 files changed, 139 insertions(+), 6 deletions(-)
Comment 5 commit-hook freebsd_committer freebsd_triage 2023-11-20 13:05:30 UTC
A commit in branch stable/14 references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=f831517d862dac2df3110c569b44e8417c3f0afa

commit f831517d862dac2df3110c569b44e8417c3f0afa
Author:     Igor Ostapenko <pm@igoro.pro>
AuthorDate: 2023-11-17 16:04:01 +0000
Commit:     Kristof Provost <kp@FreeBSD.org>
CommitDate: 2023-11-20 10:30:19 +0000

    pf: fix dummynet + ipdivert use case

    Dummynet re-injects an mbuf with MTAG_IPFW_RULE added, and the same mtag
    is used by divert(4) as parameters for packet diversion.

    If according to pf rule set a packet should go through dummynet first
    and through ipdivert after then mentioned mtag must be removed after
    dummynet not to make ipdivert think that this is its input parameters.

    At the very beginning ipfw consumes this mtag what means the same
    behavior with tag clearing after dummynet.

    And after fabf705f4b5a pf passes parameters to ipdivert using its
    personal MTAG_PF_DIVERT mtag.

    PR:             274850
    Reviewed by:    kp
    Differential Revision:  https://reviews.freebsd.org/D42609

    (cherry picked from commit fe3bb40b9e807d4010617de1ef040ba3aa623487)

 sys/netpfil/pf/pf.c               |  27 +++++++--
 tests/sys/netpfil/pf/divert-to.sh | 118 +++++++++++++++++++++++++++++++++++++-
 2 files changed, 139 insertions(+), 6 deletions(-)
Comment 6 Mark Linimon freebsd_committer freebsd_triage 2024-01-19 14:58:12 UTC
^Triage: assign to committer for evaluation of mfc-stable13.
Comment 7 Kristof Provost freebsd_committer freebsd_triage 2024-01-21 21:56:47 UTC
There's no dummynet on pf in 13, so this will not be MFC'd there.