Bug 215830 - lib/libfetch regression
Summary: lib/libfetch regression
Status: Closed Not A Bug
Alias: None
Product: Base System
Classification: Unclassified
Component: bin (show other bugs)
Version: 10.3-STABLE
Hardware: Any Any
: --- Affects Many People
Assignee: freebsd-bugs (Nobody)
URL:
Keywords: regression
Depends on:
Blocks:
 
Reported: 2017-01-06 14:02 UTC by jhs
Modified: 2017-01-18 18:32 UTC (History)
3 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description jhs 2017-01-06 14:02:28 UTC
fetch -v ftp://www.eecs.harvard.edu/pub/nr/noweb-2.11b.tgz
 Fails for jhs@berklix.com 10.3-p4 & 10.3-STABLE
 Fails for mat@freebsd.org
 Works for jhs@berklix.com 6.4-RELEASE

Detail at:
 start: https://lists.freebsd.org/pipermail/freebsd-ports/2017-January/106541.html
 end:   https://lists.freebsd.org/pipermail/freebsd-ports/2017-January/106547.html
detail there includes
 I copied 12.0-CURRENT /usr/src/usr.bin/fetch/fetch.c	to 6.4 box
 & that works, indicating fault must lie in newer src/lib/libfetch
Next thing could be
 Try test forward porting older eg 6.4 libfetch to eg 10 or current.
  Best if someone familiar with libfetch investigates, 
  + I have no time for a while (traveling)
Comment 1 John Hein 2017-01-06 16:53:00 UTC
Fails for me on 6.4 here on my network:

% fetch -v ftp://www.eecs.harvard.edu/pub/nr/noweb-2.11b.tgz
looking up www.eecs.harvard.edu
connecting to www.eecs.harvard.edu:21
setting passive mode
fetch: ftp://www.eecs.harvard.edu/pub/nr/noweb-2.11b.tgz: Connection reset by peer


... unless I disable passive mode:

% env FTP_PASSIVE_MODE=no fetch -v ftp://www.eecs.harvard.edu/pub/nr/noweb-2.11b.tgz
looking up www.eecs.harvard.edu
connecting to www.eecs.harvard.edu:21
binding data socket
initiating transfer
remote size / mtime: 738870 / 1150149960
noweb-2.11b.tgz                               100% of  721 kB  169 kBps



Disabling passive mode also works here on FB 7, 8, 9, 10 & 11.
Also when using wget & lftp, they also fail here unless forcing active mode.

Note that active mode may not work for you depending on your network - e.g., what your firewall allows.

Is it possible that something about your 6.4 box's environment has somehow triggered your ftp transaction to use non-passive mode?
Comment 2 John Hein 2017-01-06 17:09:13 UTC
Use two (or more) -v's to have fetch(1) show the ftp dialog which will give you better visibility to see what mode you are using (similar to -d with ftp(1)).
Comment 3 Dag-Erling Smørgrav freebsd_committer freebsd_triage 2017-01-17 14:52:50 UTC
The FTP server is misconfigured.  When a transfer is initiated in passive mode, it instructs the client to connect to an RFC1918 address (10.137.96.102).  It seems Harvard moved their server behind a NATing router which does not properly rewrite the PASV response.  You should contact <ftpmaster@eecs.harvard.edu> and ask them to address the issue, either by assigning a routeable address to their FTP server or by setting up a transparent proxy.
Comment 4 John Hein 2017-01-18 01:14:19 UTC
(In reply to Dag-Erling Smørgrav from comment #3)

The OP was claiming that something changed in libfetch after 6.4-RELEASE.  The broken Harvard ftp server was just his way to empirically observe that hypothesis.  However, I could not reproduce his observation that 6.4 is different in this regard.  Perhaps there is an environment difference on his end of which he was not aware.

As such, the end result (closing this bug) is the same, but the reason for closure is subtly different (it could be more specific than "not a bug", but rather "unable to reproduce" - that is, we can't reproduce his reported success on 6.4, so it appears to _not_ be a post-6.4 behavior change).
Comment 5 Dag-Erling Smørgrav freebsd_committer freebsd_triage 2017-01-18 13:17:26 UTC
Yes, something changed.  Passive mode is now the default.  And I'd like to point out that 6.4 is over 8 years old...
Comment 6 John Hein 2017-01-18 18:32:03 UTC
The default login class defined in login.conf set FTP_PASSIVE_MODE=YES long before 6.4.

===========
r47583 | jkh | 1999-05-28 05:07:16 -0600 (Fri, 28 May 1999) | 2 lines
Changed paths:
   M /head/etc/login.conf

Set FTP_PASSIVE_MODE=YES by default in the default login class.
===========

That wasn't removed from the default login class until 9.x (in 2012 months after libfetch changed to passive by default - changes by des, so this info is for the casual observer) and was not backported even to 8.x let alone 6.x.

So if it worked for the OP on 6.4, he must have changed his environment in some non-default way.  That's all I was trying to say - hoping it might clear up the mystery for the OP as to why he saw different behavior.