Bug 220468

Summary: [libfetch] is not handling 407 (proxy auth) when connecting to https using connect tunnel (patch)
Product: Base System Reporter: Egil Hasting <egil.hasting>
Component: binAssignee: Dag-Erling Smørgrav <des>
Status: Open ---    
Severity: Affects Some People CC: bapt, cem, des, egil.hasting, ev.lyapin, garga, koobs, lme
Priority: --- Keywords: needs-qa
Version: 11.0-STABLEFlags: koobs: mfc-stable10?
koobs: mfc-stable11?
garga: mfc-stable12?
Hardware: Any   
OS: Any   
Attachments:
Description Flags
patched http.c file allowing authed connect tunnel with https as a target
none
patch of the diff from freebsd 11.0 release src
none
patch of the diff from freebsd 11.0 release src none

Description Egil Hasting 2017-07-04 13:51:21 UTC
Created attachment 184056 [details]
patched http.c file allowing authed connect tunnel with https as a target

Using:
export HTTP_PROXY_AUTH="basic:*:username:password"
export HTTP_PROXY="http://<proxy_ip>:3128"


following will FAIL with 407:
fetch https://<pkgrepohost>/repo/meta.txz  

following will WORK:
fetch http://<pkgrepohost>/repo/meta.txz


this is also affecting pkgng which are compiling libfetch.
Comment 1 Egil Hasting 2017-07-04 13:54:28 UTC
Patch allows 
fetch https://<pkgrepohost>/repo/meta.txz  

to WORK, if that was not clear in previous comment.
Comment 2 Baptiste Daroussin freebsd_committer 2017-07-04 15:35:34 UTC
Can you send a patch rather than the full file patched?

diff -u http.c.orig http.c > http.c.patch

should make one for you if you don't know how to make one.
Comment 3 Egil Hasting 2017-07-04 18:15:43 UTC
Created attachment 184057 [details]
patch of the diff from freebsd 11.0 release src

Added patch on request
Comment 4 Egil Hasting 2017-07-05 11:39:30 UTC
Created attachment 184069 [details]
patch of the diff from freebsd 11.0 release src

Removed a segfault when not supplying auth information in either url or HTTP_PROXY_AUTH
improved error message and exit message on fail.
Comment 5 Eugene V. Lyapin 2017-10-18 13:44:48 UTC
We also have big troubles with fetch, no credentials are sent to remote host when CONNECT method used. Please fix it ASAP.

$ export HTTP_PROXY_AUTH='basic:*:proxy_user:PROXY_PASS'
$ export HTTP_PROXY='http://local.proxy.me:3128/'
$ export HTTPS_PROXY='http://local.proxy.me:3128/'

fetch HTTP url via PROXY:

$ fetch http://google.com -vv
scheme:   "http"
user:     ""
password: ""
host:     "google.com"
port:     "0"
document: "/"
scheme:   "http"
user:     ""
password: ""
host:     "local.proxy.me"
port:     "3128"
document: "/"
---> local.proxy.me:3128
resolving server address: local.proxy.me:3128
requesting http://google.com/
>>> GET http://google.com/ HTTP/1.1
>>> Host: google.com
>>> Accept: */*
>>> User-Agent: fetch libfetch/2.0
>>> Connection: close
>>>
<<< HTTP/1.1 407 Proxy Authentication Required
proxy requires authorization
<<< Proxy-Authenticate: NEGOTIATE
<<< Proxy-Authenticate: NTLM
<<< Proxy-Authenticate: BASIC realm="IWA3"
<<< Cache-Control: no-cache
<<< Pragma: no-cache
<<< Content-Type: text/html; charset=utf-8
<<< Proxy-Connection: close
<<< Set-Cookie: BCSI-CS-e773a25e87ae05cc=2; Path=/
<<< Connection: close
<<< Content-Length: 849
<<<
content length: [849]
---> local.proxy.me:3128
resolving server address: local.proxy.me:3128
requesting http://google.com/
>>> GET http://google.com/ HTTP/1.1
>>> Host: google.com
basic: usr: [proxy_user]
basic: pwd: [PROXY_PASS]
>>> Proxy-Authorization: Basic c3ZjX2VzbWd43m9ib3Q6SFA4X325KjkjekgsXF5jP1UwTiI=
>>> Accept: */*
>>> User-Agent: fetch libfetch/2.0
>>> Connection: close
>>>
<<< HTTP/1.1 302 Found
<<< Content-Type: text/html; charset=UTF-8
<<< Referrer-Policy: no-referrer
<<< Location: http://www.google.ru/?gfe_rd=cr&dcr=0&ei=llfnWaf3F7HG7gT7p4-gBw
<<< Content-Length: 268
302 redirect to http://www.google.ru/?gfe_rd=cr&dcr=0&ei=llfnWaf3F7HG7gT7p4-gBw
scheme:   "http"
user:     ""
password: ""
host:     "www.google.ru"
port:     "0"
document: "/?gfe_rd=cr&dcr=0&ei=llfnWaf3F7HG7gT7p4-gBw"
<<< Date: Wed, 18 Oct 2017 13:31:02 GMT
content length: [268]
<<< Cache-Control: private, proxy-revalidate
<<< Connection: close
<<<
---> local.proxy.me:3128
resolving server address: local.proxy.me:3128
requesting http://www.google.ru/?gfe_rd=cr&dcr=0&ei=llfnWaf3F7HG7gT7p4-gBw
>>> GET http://www.google.ru/?gfe_rd=cr&dcr=0&ei=llfnWaf3F7HG7gT7p4-gBw HTTP/1.1
>>> Host: www.google.ru
basic: usr: [proxy_user]
basic: pwd: [PROXY_PASS]
>>> Proxy-Authorization: Basic c3ZjX2VzbWd43m9ib3Q6SFA4X325KjkjekgsXF5jP1UwTiI=
>>> Accept: */*
>>> User-Agent: fetch libfetch/2.0
>>> Connection: close
>>>
<<< HTTP/1.1 200 OK
<<< Date: Wed, 18 Oct 2017 13:31:02 GMT
<<< Expires: -1
<<< Content-Type: text/html; charset=windows-1251
<<< P3P: CP="This is not a P3P policy! See g.co/p3phelp for more info."
<<< Server: gws
<<< X-XSS-Protection: 1; mode=block
<<< X-Frame-Options: SAMEORIGIN
<<< Accept-Ranges: none
<<< Vary: Accept-Encoding
<<< Transfer-Encoding: chunked
<<< Cache-Control: private, max-age=0, proxy-revalidate
<<< Connection: close
<<< Set-Cookie: 1P_JAR=2017-10-18-13; expires=Wed, 25-Oct-2017 13:31:02 GMT; path=/; domain=.google.ru
<<< Set-Cookie: NID=114=BN3CH2k6S-NantH3YSo7BDamqqS4zq65i3TCQfxjPtiPwJ3cWwy-Ck3uFavI_ZoDw_4Kw_5gSKNUmxZp-zowexGOC0pywbNpIIAoGX7p_-HYEWpPtDjMalnCCj9BGf8I; expires=Thu, 19-Apr-2018 13:31:02 GMT; path=/; domain=.google.ru; HttpOnly
<<<
offset 0, length -1, size -1, clength -1
fetch: http://google.com: size of remote file is not known
local size / mtime: 11314 / 1508333405
google.com                                               0  B    0  Bps<<< 2c39
http_new_chunk(): new chunk: 11321 (11321)
<<< 0
http_new_chunk(): end of last chunk
google.com                                              11 kB  134 MBps 00m00s

fetch HTTPS url via PROXY:

$ fetch https://google.com -vv
scheme:   "https"
user:     ""
password: ""
host:     "google.com"
port:     "0"
document: "/"
scheme:   "http"
user:     ""
password: ""
host:     "local.proxy.me"
port:     "3128"
document: "/"
---> local.proxy.me:3128
resolving server address: local.proxy.me:3128
>>> CONNECT google.com:443 HTTP/1.1
>>> Host: google.com:443
>>>
<<< HTTP/1.1 407 Proxy Authentication Required
fetch: https://google.com: Proxy Authentication Required
Comment 6 Conrad Meyer freebsd_committer 2017-12-28 18:03:12 UTC
DES - Ping.  Don't want this to get dropped on the floor.
Comment 7 Lars Engels freebsd_committer 2019-04-18 08:04:54 UTC
Ping again.