Bug 243299

Summary: fetch(1) fails to handle redirects on a valid URL
Product: Base System Reporter: Yuri Victorovich <yuri>
Component: binAssignee: freebsd-bugs (Nobody) <bugs>
Status: New ---    
Severity: Affects Only Me CC: pprocacci, ronald-lists
Priority: ---    
Version: 12.1-STABLE   
Hardware: Any   
OS: Any   
Attachments:
Description Flags
Add support for relative urls in redirect responses
none
Updated patch to support redirected relative urls to fetch. none

Description Yuri Victorovich freebsd_committer freebsd_triage 2020-01-12 19:42:01 UTC 
> $ fetch "https://www.tddft.org/programs/libxc/down.php?file=4.3.4/libxc-4.3.4.tar.gz"
> fetch: https://www.tddft.org/programs/libxc/down.php?file=4.3.4/libxc-4.3.4.tar.gz: No address record


But wget succeeds:
> $ wget "https://www.tddft.org/programs/libxc/down.php?file=4.3.4/libxc-4.3.4.tar.gz"
> --2020-01-12 11:40:57--  https://www.tddft.org/programs/libxc/down.php?file=4.3.4/libxc-4.3.4.tar.gz
> Resolving www.tddft.org (www.tddft.org)... 193.137.208.200
> Connecting to www.tddft.org (www.tddft.org)|193.137.208.200|:443... connected.
> HTTP request sent, awaiting response... 302 Found
> Location: down/4.3.4/libxc-4.3.4.tar.gz [following]
> --2020-01-12 11:40:59--  https://www.tddft.org/programs/libxc/down/4.3.4/libxc-4.3.4.tar.gz
> Reusing existing connection to www.tddft.org:443.
> HTTP request sent, awaiting response... 200 OK
> Length: 15602575 (15M) [application/x-gzip]
> Saving to: ‘down.php?file=4.3.4%2Flibxc-4.3.4.tar.gz.4’
>
> down.php?file=4.3.4%2Flibxc-4.3.4.tar.gz.4    1%[>                                                                                        ] 279.68K   187KB/s               ^
Comment 1 Ronald Klop 2022-08-03 13:22:41 UTC 
To clarify the issue:

$ fetch -v "https://www.tddft.org/programs/libxc/down.php?file=4.3.4/libxc-4.3.4.tar.gz"
resolving server address: www.tddft.org:443
SSL options: 82004854
Peer verification enabled
Using CA cert file: /usr/local/etc/ssl/cert.pem
Verify hostname
TLSv1.2 connection established using ECDHE-RSA-AES256-GCM-SHA384
Certificate subject: /CN=tddft.org
Certificate issuer: /C=US/O=Let's Encrypt/CN=R3
requesting https://www.tddft.org/programs/libxc/down.php?file=4.3.4/libxc-4.3.4.tar.gz
302 redirect to down/4.3.4/libxc-4.3.4.tar.gz
resolving server address: down:0
fetch: https://www.tddft.org/programs/libxc/down.php?file=4.3.4/libxc-4.3.4.tar.gz: No address record

Fetch tries "down" as the server name of the next hop.
Comment 2 pprocacci 2022-08-21 06:42:06 UTC 
Created attachment 236031 [details]
Add support for relative urls in redirect responses

RFC 7231 has support for relative urls.
I saw this bug report and had to check.  I didn't believe it.

The attach patch fixes the op's issue, but truly it needs to be tested with all kinds of responses.

Additionally, I program in C in a much much different way than the source file's layout.  Safe to apply, sure, but it'd be wise for someone else to give it a once over to ensure it conforms to your standards; something someone else can do if they so choose.
Comment 3 pprocacci 2022-08-21 06:46:49 UTC 
Created attachment 236032 [details]
Updated patch to support redirected relative urls to fetch.

Just added the check to the return of the function ....  Hey it's 3am ... sheesh!