Bug 220468 - [libfetch] is not handling 407 (proxy auth) when connecting to https using connect tunnel (patch)
Summary: [libfetch] is not handling 407 (proxy auth) when connecting to https using co...
Status: Open
Alias: None
Product: Base System
Classification: Unclassified
Component: bin (show other bugs)
Version: 11.0-STABLE
Hardware: Any Any
: --- Affects Some People
Assignee: Dag-Erling Smørgrav
URL:
Keywords: needs-qa
Depends on:
Blocks:
 
Reported: 2017-07-04 13:51 UTC by Egil Hasting
Modified: 2018-11-15 13:51 UTC (History)
7 users (show)

See Also:
koobs: mfc-stable10?
koobs: mfc-stable11?
garga: mfc-stable12?


Attachments
patched http.c file allowing authed connect tunnel with https as a target (52.13 KB, patch)
2017-07-04 13:51 UTC, Egil Hasting
no flags Details | Diff
patch of the diff from freebsd 11.0 release src (3.28 KB, patch)
2017-07-04 18:15 UTC, Egil Hasting
no flags Details | Diff
patch of the diff from freebsd 11.0 release src (3.48 KB, patch)
2017-07-05 11:39 UTC, Egil Hasting
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Egil Hasting 2017-07-04 13:51:21 UTC
Created attachment 184056 [details]
patched http.c file allowing authed connect tunnel with https as a target

Using:
export HTTP_PROXY_AUTH="basic:*:username:password"
export HTTP_PROXY="http://<proxy_ip>:3128"


following will FAIL with 407:
fetch https://<pkgrepohost>/repo/meta.txz  

following will WORK:
fetch http://<pkgrepohost>/repo/meta.txz


this is also affecting pkgng which are compiling libfetch.
Comment 1 Egil Hasting 2017-07-04 13:54:28 UTC
Patch allows 
fetch https://<pkgrepohost>/repo/meta.txz  

to WORK, if that was not clear in previous comment.
Comment 2 Baptiste Daroussin freebsd_committer 2017-07-04 15:35:34 UTC
Can you send a patch rather than the full file patched?

diff -u http.c.orig http.c > http.c.patch

should make one for you if you don't know how to make one.
Comment 3 Egil Hasting 2017-07-04 18:15:43 UTC
Created attachment 184057 [details]
patch of the diff from freebsd 11.0 release src

Added patch on request
Comment 4 Egil Hasting 2017-07-05 11:39:30 UTC
Created attachment 184069 [details]
patch of the diff from freebsd 11.0 release src

Removed a segfault when not supplying auth information in either url or HTTP_PROXY_AUTH
improved error message and exit message on fail.
Comment 5 Eugene V. Lyapin 2017-10-18 13:44:48 UTC
We also have big troubles with fetch, no credentials are sent to remote host when CONNECT method used. Please fix it ASAP.

$ export HTTP_PROXY_AUTH='basic:*:proxy_user:PROXY_PASS'
$ export HTTP_PROXY='http://local.proxy.me:3128/'
$ export HTTPS_PROXY='http://local.proxy.me:3128/'

fetch HTTP url via PROXY:

$ fetch http://google.com -vv
scheme:   "http"
user:     ""
password: ""
host:     "google.com"
port:     "0"
document: "/"
scheme:   "http"
user:     ""
password: ""
host:     "local.proxy.me"
port:     "3128"
document: "/"
---> local.proxy.me:3128
resolving server address: local.proxy.me:3128
requesting http://google.com/
>>> GET http://google.com/ HTTP/1.1
>>> Host: google.com
>>> Accept: */*
>>> User-Agent: fetch libfetch/2.0
>>> Connection: close
>>>
<<< HTTP/1.1 407 Proxy Authentication Required
proxy requires authorization
<<< Proxy-Authenticate: NEGOTIATE
<<< Proxy-Authenticate: NTLM
<<< Proxy-Authenticate: BASIC realm="IWA3"
<<< Cache-Control: no-cache
<<< Pragma: no-cache
<<< Content-Type: text/html; charset=utf-8
<<< Proxy-Connection: close
<<< Set-Cookie: BCSI-CS-e773a25e87ae05cc=2; Path=/
<<< Connection: close
<<< Content-Length: 849
<<<
content length: [849]
---> local.proxy.me:3128
resolving server address: local.proxy.me:3128
requesting http://google.com/
>>> GET http://google.com/ HTTP/1.1
>>> Host: google.com
basic: usr: [proxy_user]
basic: pwd: [PROXY_PASS]
>>> Proxy-Authorization: Basic c3ZjX2VzbWd43m9ib3Q6SFA4X325KjkjekgsXF5jP1UwTiI=
>>> Accept: */*
>>> User-Agent: fetch libfetch/2.0
>>> Connection: close
>>>
<<< HTTP/1.1 302 Found
<<< Content-Type: text/html; charset=UTF-8
<<< Referrer-Policy: no-referrer
<<< Location: http://www.google.ru/?gfe_rd=cr&dcr=0&ei=llfnWaf3F7HG7gT7p4-gBw
<<< Content-Length: 268
302 redirect to http://www.google.ru/?gfe_rd=cr&dcr=0&ei=llfnWaf3F7HG7gT7p4-gBw
scheme:   "http"
user:     ""
password: ""
host:     "www.google.ru"
port:     "0"
document: "/?gfe_rd=cr&dcr=0&ei=llfnWaf3F7HG7gT7p4-gBw"
<<< Date: Wed, 18 Oct 2017 13:31:02 GMT
content length: [268]
<<< Cache-Control: private, proxy-revalidate
<<< Connection: close
<<<
---> local.proxy.me:3128
resolving server address: local.proxy.me:3128
requesting http://www.google.ru/?gfe_rd=cr&dcr=0&ei=llfnWaf3F7HG7gT7p4-gBw
>>> GET http://www.google.ru/?gfe_rd=cr&dcr=0&ei=llfnWaf3F7HG7gT7p4-gBw HTTP/1.1
>>> Host: www.google.ru
basic: usr: [proxy_user]
basic: pwd: [PROXY_PASS]
>>> Proxy-Authorization: Basic c3ZjX2VzbWd43m9ib3Q6SFA4X325KjkjekgsXF5jP1UwTiI=
>>> Accept: */*
>>> User-Agent: fetch libfetch/2.0
>>> Connection: close
>>>
<<< HTTP/1.1 200 OK
<<< Date: Wed, 18 Oct 2017 13:31:02 GMT
<<< Expires: -1
<<< Content-Type: text/html; charset=windows-1251
<<< P3P: CP="This is not a P3P policy! See g.co/p3phelp for more info."
<<< Server: gws
<<< X-XSS-Protection: 1; mode=block
<<< X-Frame-Options: SAMEORIGIN
<<< Accept-Ranges: none
<<< Vary: Accept-Encoding
<<< Transfer-Encoding: chunked
<<< Cache-Control: private, max-age=0, proxy-revalidate
<<< Connection: close
<<< Set-Cookie: 1P_JAR=2017-10-18-13; expires=Wed, 25-Oct-2017 13:31:02 GMT; path=/; domain=.google.ru
<<< Set-Cookie: NID=114=BN3CH2k6S-NantH3YSo7BDamqqS4zq65i3TCQfxjPtiPwJ3cWwy-Ck3uFavI_ZoDw_4Kw_5gSKNUmxZp-zowexGOC0pywbNpIIAoGX7p_-HYEWpPtDjMalnCCj9BGf8I; expires=Thu, 19-Apr-2018 13:31:02 GMT; path=/; domain=.google.ru; HttpOnly
<<<
offset 0, length -1, size -1, clength -1
fetch: http://google.com: size of remote file is not known
local size / mtime: 11314 / 1508333405
google.com                                               0  B    0  Bps<<< 2c39
http_new_chunk(): new chunk: 11321 (11321)
<<< 0
http_new_chunk(): end of last chunk
google.com                                              11 kB  134 MBps 00m00s

fetch HTTPS url via PROXY:

$ fetch https://google.com -vv
scheme:   "https"
user:     ""
password: ""
host:     "google.com"
port:     "0"
document: "/"
scheme:   "http"
user:     ""
password: ""
host:     "local.proxy.me"
port:     "3128"
document: "/"
---> local.proxy.me:3128
resolving server address: local.proxy.me:3128
>>> CONNECT google.com:443 HTTP/1.1
>>> Host: google.com:443
>>>
<<< HTTP/1.1 407 Proxy Authentication Required
fetch: https://google.com: Proxy Authentication Required
Comment 6 Conrad Meyer freebsd_committer 2017-12-28 18:03:12 UTC
DES - Ping.  Don't want this to get dropped on the floor.