Bug 256917

Summary: ftp-proxy doesn't work properly
Product: Base System Reporter: Kan Sasaki <sasaki12>
Component: binAssignee: Kristof Provost <kp>
Status: Closed FIXED    
Severity: Affects Only Me CC: kp
Priority: ---    
Version: 12.2-STABLE   
Hardware: amd64   
OS: Any   

Description Kan Sasaki 2021-07-01 08:27:23 UTC
stable/12-n233341-43a58daffe72

I have recently updated my server to the above. Since then, ftp-proxy doesn't work properly.
When I connect to the ftp server, login succeeds. However, when I type ls, the connection is closed.

# ftp xxx
Connected to xxx
220 XXX FTP server ready
Name (xxx:xxx):
331 Please specify the password.
Password: 
230 Login successful.
Remote system type is UNIX.
Using binary mode to transfer files.
ftp> ls
421 Service not available, remote server has closed connection.
257 "/" is the current directory


The result of truss -p PID_OF_FTP_PROXY after typing ls is as follows.

kevent(6,{ },0,{ 7,EVFILT_READ,EV_ONESHOT,0,0x6,0x8014dd1c0 },64,{ 86399.998605000 }) = 1 (0x1)
ioctl(7,FIONREAD,0x7fffffffe6ec)                 = 0 (0x0)
read(7,"EPSV\r\n",6)                             = 6 (0x6)
kevent(6,{ 7,EVFILT_READ,EV_DELETE,0,0,0x0 7,EVFILT_READ,EV_ADD|EV_ONESHOT,0,0,0x8014dd1c0 8,EVFILT_WRITE,EV_ADD|EV_ONESHOT,0,0,0x8014dd098 },3,{ 7,EVFILT_READ,EV_ERROR,0,0x2,0x0 },64,{ 86394.255447000 }) = 1 (0x1)
kevent(6,{ },0,{ 8,EVFILT_WRITE,EV_ONESHOT,0,0x832c,0x8014dd098 },64,{ 86394.255383000 }) = 1 (0x1)
write(8,"EPSV\r\n",6)                            = 6 (0x6)
kevent(6,{ 8,EVFILT_WRITE,EV_DELETE,0,0,0x0 },1,{ 8,EVFILT_WRITE,EV_ERROR,0,0x2,0x0 },64,{ 86394.255268000 }) = 1 (0x1)
kevent(6,{ },0,{ 8,EVFILT_READ,EV_ONESHOT,0,0x30,0x8014dd000 },64,{ 86394.255214000 }) = 1 (0x1)
ioctl(8,FIONREAD,0x7fffffffe6ec)                 = 0 (0x0)
read(8,"229 Entering Extended Passive Mo"...,48) = 48 (0x30)
getpid()                                         = 7802 (0x1e7a)
ioctl(4,DIOCXBEGIN,0x102d958)                    = 0 (0x0)
getpid()                                         = 7802 (0x1e7a)
ioctl(4,DIOCBEGINADDRS,0x102d4e8)                = 0 (0x0)
ioctl(4,DIOCADDRULENV,0x7fffffffe560)            ERR#22 'Invalid argument'
__sysctl("kern.hostname",2,0x7fffffffe0c0,0x7fffffffd368,0x0,0) = 0 (0x0)
getpid()                                         = 7802 (0x1e7a)
sendto(5,"<26>1 2021-07-01T17:22:03.310854"...,129,0,NULL,0) = 129 (0x81)
ioctl(4,DIOCXROLLBACK,0x102d958)                 = 0 (0x0)
write(7,0x8014e4000,0)                           = 0 (0x0)
write(8,0x8014e4200,0)                           = 0 (0x0)
close(7)                                         = 0 (0x0)
close(8)                                         = 0 (0x0)
getpid()                                         = 7802 (0x1e7a)
ioctl(4,DIOCXBEGIN,0x102d958)                    = 0 (0x0)
ioctl(4,DIOCXCOMMIT,0x102d958)                   = 0 (0x0)
kevent(6,{ 8,EVFILT_READ,EV_DELETE,0,0,0x0 8,EVFILT_READ,EV_ADD|EV_ONESHOT,0,0,0x8014dd000 7,EVFILT_READ,EV_DELETE,0,0,0x0 8,EVFILT_READ,EV_DELETE,0,0,0x0 },4,{ 8,EVFILT_READ,EV_ERROR,0,0x9,0x0 8,EVFILT_READ,EV_ERROR,0,0x9,0x8014dd000 7,EVFILT_READ,EV_ERROR,0,0x9,0x0 8,EVFILT_READ,EV_ERROR,0,0x9,0x0 },64,{ 5.000000000 }) = 4 (0x4)
kevent(6,{ },0,{ },64,{ 5.000000000 })           = 0 (0x0)


I changed the source back to 0d71f9f36e6c and the issue has gone. I think the following commit is causing this issue.

commit 95be9288f01f30a50440ea56d11468a2c6e18fed (HEAD)
Author: Kristof Provost <kp@FreeBSD.org>
Date:   Mon Mar 29 14:03:39 2021 +0200

    (t)ftp-proxy: use libpfctl

    Reviewed by:    glebius
    MFC after:      4 weeks
    Sponsored by:   Rubicon Communications, LLC ("Netgate")
    Differential Revision:  https://reviews.freebsd.org/D29641
Comment 1 Kristof Provost freebsd_committer freebsd_triage 2021-07-01 08:37:10 UTC
Can you test this: 

diff --git a/contrib/pf/ftp-proxy/filter.c b/contrib/pf/ftp-proxy/filter.c
index dad6324808bc..e4787985e99f 100644
--- a/contrib/pf/ftp-proxy/filter.c
+++ b/contrib/pf/ftp-proxy/filter.c
@@ -103,8 +103,7 @@ add_nat(u_int32_t id, struct sockaddr *src, struct sockaddr *dst,
                    &satosin6(nat)->sin6_addr.s6_addr, 16);
                memset(&pfp.addr.addr.v.a.mask.addr8, 255, 16);
        }
-       if (pfctl_add_rule(dev, &pfrule, pfanchor, pfanchor_call,
-           pfticket, pfpool_ticket))
+       if (ioctl(dev, DIOCADDADDR, &pfp) == -1)
                return (-1);

        pfrule.rpool.proxy_port[0] = nat_range_low;
@@ -138,8 +137,7 @@ add_rdr(u_int32_t id, struct sockaddr *src, struct sockaddr *dst,
                    &satosin6(rdr)->sin6_addr.s6_addr, 16);
                memset(&pfp.addr.addr.v.a.mask.addr8, 255, 16);
        }
-       if (pfctl_add_rule(dev, &pfrule, pfanchor, pfanchor_call,
-           pfticket, pfpool_ticket))
+       if (ioctl(dev, DIOCADDADDR, &pfp) == -1)
                return (-1);

        pfrule.rpool.proxy_port[0] = rdr_port;
Comment 2 Kan Sasaki 2021-07-01 08:57:52 UTC
(In reply to Kristof Provost from comment #1)

Yes. But the result is no good.

230 Login successful.
Remote system type is UNIX.
Using binary mode to transfer files.
ftp> ls
229 Entering Extended Passive Mode (|||55485|)
(stopping here)
Comment 3 commit-hook freebsd_committer freebsd_triage 2021-07-01 19:36:16 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=dd82fd3543022017b84007ac1a0d45fc683f9850

commit dd82fd3543022017b84007ac1a0d45fc683f9850
Author:     Kristof Provost <kp@FreeBSD.org>
AuthorDate: 2021-07-01 15:15:36 +0000
Commit:     Kristof Provost <kp@FreeBSD.org>
CommitDate: 2021-07-01 19:34:40 +0000

    pf tests: ftp-proxy test

    Basic test case for ftp-proxy

    PR:             256917
    MFC after:      1 week
    Sponsored by:   Rubicon Communications, LLC ("Netgate")

 tests/sys/netpfil/pf/Makefile       |  1 +
 tests/sys/netpfil/pf/proxy.sh (new) | 95 +++++++++++++++++++++++++++++++++++++
 2 files changed, 96 insertions(+)
Comment 4 commit-hook freebsd_committer freebsd_triage 2021-07-01 19:36:18 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=8923ea6c867fd75b08b76883ec122c429a4018f9

commit 8923ea6c867fd75b08b76883ec122c429a4018f9
Author:     Kristof Provost <kp@FreeBSD.org>
AuthorDate: 2021-07-01 15:16:10 +0000
Commit:     Kristof Provost <kp@FreeBSD.org>
CommitDate: 2021-07-01 19:34:40 +0000

    ftp-proxy: Revert incorrect migration to libpfctl

    libpfctl supports creating rules, but not (yet) adding addresses to a
    pool. Adding addresses certainly does not work through adding a rule.

    PR:             256917
    MFC after:      1 week
    Sponsored by:   Rubicon Communications, LLC ("Netgate")

 contrib/pf/ftp-proxy/filter.c | 6 ++----
 1 file changed, 2 insertions(+), 4 deletions(-)
Comment 5 Kristof Provost freebsd_committer freebsd_triage 2021-07-01 19:37:28 UTC
(In reply to Kan Sasaki from comment #2)
I'm going to need more details on your setup and what goes wrong, because I've tested the fix and added a test case for ftp-proxy. With the below fix it just works. 

I've just committed both fix and test to main, and I'd expect it to be final.
Comment 6 Kan Sasaki 2021-07-02 02:45:51 UTC
(In reply to Kristof Provost from comment #5)
I've done something wrong during tests. I did patch, buildworld and installworld again and it worked fine.

Sorry for noise.
Comment 7 commit-hook freebsd_committer freebsd_triage 2021-07-08 11:50:53 UTC
A commit in branch stable/13 references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=f6d448caf69889eca9e41a0815db65a22fe7652d

commit f6d448caf69889eca9e41a0815db65a22fe7652d
Author:     Kristof Provost <kp@FreeBSD.org>
AuthorDate: 2021-07-01 15:15:36 +0000
Commit:     Kristof Provost <kp@FreeBSD.org>
CommitDate: 2021-07-08 06:47:09 +0000

    pf tests: ftp-proxy test

    Basic test case for ftp-proxy

    PR:             256917
    MFC after:      1 week
    Sponsored by:   Rubicon Communications, LLC ("Netgate")

    (cherry picked from commit dd82fd3543022017b84007ac1a0d45fc683f9850)

 tests/sys/netpfil/pf/Makefile       |  1 +
 tests/sys/netpfil/pf/proxy.sh (new) | 95 +++++++++++++++++++++++++++++++++++++
 2 files changed, 96 insertions(+)
Comment 8 commit-hook freebsd_committer freebsd_triage 2021-07-08 11:50:54 UTC
A commit in branch stable/13 references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=4f5a2009ad8ad98a457ddecb63fe1ed8a968226d

commit 4f5a2009ad8ad98a457ddecb63fe1ed8a968226d
Author:     Kristof Provost <kp@FreeBSD.org>
AuthorDate: 2021-07-01 15:16:10 +0000
Commit:     Kristof Provost <kp@FreeBSD.org>
CommitDate: 2021-07-08 06:46:54 +0000

    ftp-proxy: Revert incorrect migration to libpfctl

    libpfctl supports creating rules, but not (yet) adding addresses to a
    pool. Adding addresses certainly does not work through adding a rule.

    PR:             256917
    MFC after:      1 week
    Sponsored by:   Rubicon Communications, LLC ("Netgate")

    (cherry picked from commit 8923ea6c867fd75b08b76883ec122c429a4018f9)

 contrib/pf/ftp-proxy/filter.c | 6 ++----
 1 file changed, 2 insertions(+), 4 deletions(-)
Comment 9 commit-hook freebsd_committer freebsd_triage 2021-07-08 11:50:58 UTC
A commit in branch stable/12 references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=e26de428fa542c0fd62a696e2d721ff6f8dbc5db

commit e26de428fa542c0fd62a696e2d721ff6f8dbc5db
Author:     Kristof Provost <kp@FreeBSD.org>
AuthorDate: 2021-07-01 15:15:36 +0000
Commit:     Kristof Provost <kp@FreeBSD.org>
CommitDate: 2021-07-08 06:49:56 +0000

    pf tests: ftp-proxy test

    Basic test case for ftp-proxy

    PR:             256917
    MFC after:      1 week
    Sponsored by:   Rubicon Communications, LLC ("Netgate")

    (cherry picked from commit dd82fd3543022017b84007ac1a0d45fc683f9850)

 tests/sys/netpfil/pf/Makefile       |  1 +
 tests/sys/netpfil/pf/proxy.sh (new) | 95 +++++++++++++++++++++++++++++++++++++
 2 files changed, 96 insertions(+)
Comment 10 commit-hook freebsd_committer freebsd_triage 2021-07-08 11:50:59 UTC
A commit in branch stable/12 references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=d442d97b7632628ac3cf3e591ddbd8b872c14818

commit d442d97b7632628ac3cf3e591ddbd8b872c14818
Author:     Kristof Provost <kp@FreeBSD.org>
AuthorDate: 2021-07-01 15:16:10 +0000
Commit:     Kristof Provost <kp@FreeBSD.org>
CommitDate: 2021-07-08 06:49:56 +0000

    ftp-proxy: Revert incorrect migration to libpfctl

    libpfctl supports creating rules, but not (yet) adding addresses to a
    pool. Adding addresses certainly does not work through adding a rule.

    PR:             256917
    MFC after:      1 week
    Sponsored by:   Rubicon Communications, LLC ("Netgate")

    (cherry picked from commit 8923ea6c867fd75b08b76883ec122c429a4018f9)

 contrib/pf/ftp-proxy/filter.c | 6 ++----
 1 file changed, 2 insertions(+), 4 deletions(-)