Bug 246984 - lang/python* Fix CVE-2020-8492, CVE-2019-18348
Summary: lang/python* Fix CVE-2020-8492, CVE-2019-18348
Status: Closed FIXED
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: Normal Affects Many People
Assignee: Danilo G. Baio
URL: https://bugs.python.org/issue39503
Keywords: security
: 246808 (view as bug list)
Depends on:
Blocks:
 
Reported: 2020-06-04 15:35 UTC by Dani
Modified: 2020-06-22 11:31 UTC (History)
11 users (show)

See Also:
dbaio: maintainer-feedback+
dbaio: merge-quarterly+


Attachments
Fix CVE-2020-8492 (3.00 KB, patch)
2020-06-04 15:36 UTC, Dani
no flags Details | Diff
python-CVE-2019-18348_CVE-2020-8492.patch (8.38 KB, patch)
2020-06-07 01:56 UTC, Danilo G. Baio
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Dani 2020-06-04 15:35:38 UTC
CVE-2020-8492 is open for quite a long time and hasen't been patched in a release except for python 3.8. This pr fixes the CVE for Python 3.6 and 3.7 and corrects/updates the wrong vuxml entries.

Please also see: https://python-security.readthedocs.io/vuln/urllib-basic-auth-regex.html

lang/python36:
  - Backport fix for CVE-2020-8492
  - Python Bug 39503: https://bugs.python.org/issue39503
  - Commit: https://github.com/python/cpython/commit/69cdeeb93e0830004a495ed854022425b93b3f3e

lang/python37:
  - Backport fix for CVE-2020-8492
  - Python Bug 39503: https://bugs.python.org/issue39503
  - Commit: https://github.com/python/cpython/commit/b57a73694e26e8b2391731b5ee0b1be59437388e

security/vuxml:
  - Update the entry for python36 to the corrected version
  - Correct the entry for python37 to the correct version, 3.7.7 does NOT have the fix included. See: https://python-security.readthedocs.io/vuln/urllib-basic-auth-regex.html
Comment 1 Dani 2020-06-04 15:36:56 UTC
Created attachment 215230 [details]
Fix CVE-2020-8492
Comment 2 Dani 2020-06-04 15:39:31 UTC
- No python39 in portstree, so no patch needed.
- No backport for python35 yet, so not patched until new release/backport released.
Comment 3 Kubilay Kocak freebsd_committer freebsd_triage 2020-06-07 00:18:26 UTC
Thank you for the report and patches Dani

Do any of the upstream 3.6 / 3.7 / head patches apply cleanly to the 3.5 port?
Comment 4 Danilo G. Baio freebsd_committer 2020-06-07 01:43:05 UTC
Hi.

Taking a look at this PR I noticed we have issues in CVE-2019-18348 as well.

And vuxml is currently wrong in both CVE's.

Simple table to explain:
-------------------------------------------------------------------------------
  2.7: 2.7.18         April 20, 2020   CVE-2019-18348 OK  /  CVE-2020-8492 OK
  3.5: 3.5.9          Nov. 2, 2019     CVE-2019-18348 MS  /  CVE-2020-8492 MS
  3.6: 3.6.9 (3.6.10) July 2, 2019     CVE-2019-18348 NR  /  CVE-2020-8492 NR
  3.7: 3.7.7          March 10, 2020   CVE-2019-18348 NR  /  CVE-2020-8492 NR   
  3.8: 3.8.3          May 13, 2020     CVE-2019-18348 OK  /  CVE-2020-8492 OK

  MS - Missing commit in upstream branch (PR open)
  NR - Next Release, commit is in the branch
-------------------------------------------------------------------------------

So we have to patch Python 3.7, update Python 3.6 to 3.6.10+patch and patch Python 3.5 for both CVE's.

And fix vuxml ASAP:
 CVE-2019-18348, needs to add 3.5, 3.6 and 3.7 packages, they are all affected in this moment.
 CVE-2020-8492,  3.7, needs to update the range, it's informing that 3.7.7 is not affected.

There is a misunderstanding about CVE-2020-8492, in the CVE text it says "3.7 through 3.7.6", but they applied the fix after 3.7.7 and it's on the branch waiting next release.

 https://python-security.readthedocs.io/vuln/urlopen-host-http-header-injection.html  (CVE-2019-18348)
 https://python-security.readthedocs.io/vuln/urllib-basic-auth-regex.html   (CVE-2020-8492)

3.5 - https://github.com/python/cpython/pull/19300  (CVE-2019-18348) PR open
3.5 - https://github.com/python/cpython/pull/19305  (CVE-2020-8492)  PR open

Both patches for 3.5 applied cleanly, but the PRs are still open, should we test it and already add to the ports tree?

So in addition to Dani's patch, we need to also address CVE-2019-18348, I think we can do this together.
Comment 5 Danilo G. Baio freebsd_committer 2020-06-07 01:56:13 UTC
Created attachment 215304 [details]
python-CVE-2019-18348_CVE-2020-8492.patch

Patch for review.

Needs to decide if we will push Python 3.5 patches here, with the pending PRs.

Could we ask for an exp-run and decide it later?
Comment 6 commit-hook freebsd_committer 2020-06-07 02:21:00 UTC
A commit references this bug:

Author: dbaio
Date: Sun Jun  7 02:20:41 UTC 2020
New revision: 538142
URL: https://svnweb.freebsd.org/changeset/ports/538142

Log:
  security/vuxml: Update CVE-2019-18348 and CVE-2020-8492 entries

  CVE-2019-18348:	Add missing Python packages range
  CVE-2020-8492:	Fix Python 3.7 entrie, it's currently affected.

  After committing fixes, we'll need to change ranges again.

  PR:		246984

Changes:
  head/security/vuxml/vuln.xml
Comment 7 Danilo G. Baio freebsd_committer 2020-06-07 12:00:31 UTC
attachment 215304 [details]
Comment 8 Antoine Brodin freebsd_committer 2020-06-07 12:10:18 UTC
I don't think this needs an exp-run
Comment 9 Danilo G. Baio freebsd_committer 2020-06-07 14:08:37 UTC
(In reply to Antoine Brodin from comment #8)

Thanks for the feedback antoine@.



Tests:

poudriere ok (11, 12, CURRENT; i386, amd64)

make test:

lang/python36:
  make test (CURRENT, 12): 

  - 381 tests OK. 
  - 2 tests failed:
      test_distutils test_posix 
  - 22 tests skipped

  No changes.

lang/python37:
  make test (CURRENT, 12):

  - 393 tests OK.
  - 3 tests failed:
      test_capi test_distutils test_posix
  - 20 tests skipped

  No changes.

lang/python35:
  make test (CURRENT, 12):

  - Ran 279 tests in 121.413s
  - FAILED (failures=2, errors=2, skipped=19)

  No changes.



Waiting review from others in python@
Comment 10 Dani 2020-06-08 06:43:23 UTC
(In reply to Danilo G. Baio from comment #9)
Thanks for taking a deep-lock at it and creating a new patch! Built and testen on FBSD 11.3 - looks good to me.
Comment 11 Kubilay Kocak freebsd_committer freebsd_triage 2020-06-10 04:09:50 UTC
*** Bug 246808 has been marked as a duplicate of this bug. ***
Comment 12 Kubilay Kocak freebsd_committer freebsd_triage 2020-06-10 04:18:25 UTC
@Dani/Danilo 

I've closed bug 246808 (earlier issue for 3.6 update for CVE) as a dupe of this issue as this one contains a superset of updates/changes.

Can we please:

- Obsolete any patches that are not relevant or superseded 

- Understand/Assess/explain the apparent difference between the upstream commit references used in bug 246808 for 3.6.10 vs this issues commit hashes for 3.6.10:


< 69cdeeb93e0830004a495ed854022425b93b3f3e.patch:-p1 (this bug)
< 83fc70159b24f5b11a5ef87c9b05c2cf4c7faeba.patch:-p1 (this bug)

> 0f10ef077fc32b60cb07780ea7234516950d0f9e.patch:-p1 (other bug)

- Summarise the update rationale, something like (but making sure its correct, because its not obvious at the moment), the following:

3.5: backport 3.<x> commits (no releases anticipated for this version)
3.6: update to 3.6.10 (covers all outstanding CVE's)
3.7: backport 3.7 patches (3.7.8 not yet released? wont be released?)
3.8 not vulnerable
Comment 13 Dani 2020-06-10 07:57:36 UTC
(In reply to Kubilay Kocak from comment #12)

Hi koobs, thanks for your feedback.

- Patch "Fix CVE-2020-8492" can be marked obsolate due to the patch of Danilo.

- bug #246808 used the commit which has been made in the Git "master"-Branch. The commits Danilo and i used, were the ones that have "specially" been made/backported to the different releases (eg. 3.6, 3.7, 3.5). See section "Timeline": https://python-security.readthedocs.io/vuln/urllib-basic-auth-regex.html

- The summary can best be done by Danilo i guess. What's basically important is:
  - A new version of Python 3.8 has been released, which fixed all open CVE's (v 3.8.3)
  - No new version released !yet! for: Python 3.5, 3.6, 3.7 
  - CVE-2019-18348 has a fix ready and merged for all python versions: https://bugs.python.org/issue38576
  - CVE-2020-8492 has a fiy ready and merged for python 3.6, 3.7, 3.8, 3.9 (https://bugs.python.org/issue39503) but not for 3.5 (https://github.com/python/cpython/pull/19305)
Comment 14 Danilo G. Baio freebsd_committer 2020-06-10 12:37:22 UTC
Thanks Dani for the explanations.

Thinking in separate commits because we have an update in the middle (Python 3.6) and Python 3.5 fixes are awaiting review from Python Core. If something happens, it will be easy to revert.

koobs@ as I know you like to organize commits, here it goes, any changes are welcome.

-------------------------------------------------------------------------------
lang/python35: Fix security issues

There are no plans for a next release of Python 3.5.

PR:   246984
Security: ca595a25-91d8-11ea-b470-080027846a02 (CVE-2019-18348)
Security: a27b0bb6-84fc-11ea-b5b4-641c67a117d8 (CVE-2020-8492)
MFH:  2020Q2
Obtained from:  https://github.com/python/cpython/pull/19300
    https://github.com/python/cpython/pull/19305.

-------------------------------------------------------------------------------
lang/python36: Update to 3.6.10, Fix security issues

The patches for CVE-2019-18348 and CVE-2020-8492 are in the 3.6 branch
and will be present on the next release.

Patch for applying CVE-2020-8492 fix here in the ports tree was reported
and submitted by Mike Fisher <mfisher911@gmail.com> and
Dani <i.dani@outlook.com>.

PR:   246984
Security: ca595a25-91d8-11ea-b470-080027846a02 (CVE-2019-18348)
Security: a27b0bb6-84fc-11ea-b5b4-641c67a117d8 (CVE-2020-8492)
MFH:  2020Q2

-------------------------------------------------------------------------------
lang/python37: Fix security issues

The patches for CVE-2019-18348 and CVE-2020-8492 are in the 3.7 branch
and will be present on the next release.

Patch for applying CVE-2020-8492 fix here in the ports tree was reported
and submitted by Dani <i.dani@outlook.com>.

PR:   246808
Security: ca595a25-91d8-11ea-b470-080027846a02 (CVE-2019-18348)
Security: a27b0bb6-84fc-11ea-b5b4-641c67a117d8 (CVE-2020-8492)
MFH:  2020Q2
X-MFH-with: 536776

-------------------------------------------------------------------------------

About https://github.com/python/cpython/pull/19300 and
https://github.com/python/cpython/pull/19305.

I subscribed on those PRs and will be watching for any changes.

After commits, vuxml will be updated.
Comment 15 Dani 2020-06-10 13:16:23 UTC
(In reply to Danilo G. Baio from comment #14)

Yeah, i think spliting up is a good idea! Thanks for your work! LGTM :)
Comment 16 commit-hook freebsd_committer 2020-06-13 13:24:46 UTC
A commit references this bug:

Author: dbaio
Date: Sat Jun 13 13:24:30 UTC 2020
New revision: 538669
URL: https://svnweb.freebsd.org/changeset/ports/538669

Log:
  lang/python36: Update to 3.6.10, Fix security issues

  The patches for CVE-2019-18348 and CVE-2020-8492 are in the 3.6 branch
  and will be present on the next release.

  Patch for applying CVE-2020-8492 fix here in the ports tree was reported
  and submitted by Mike Fisher <mfisher911@gmail.com> and
  Dani <i.dani@outlook.com>.

  PR:		246984
  MFH:		2020Q2
  Security:	ca595a25-91d8-11ea-b470-080027846a02 (CVE-2019-18348)
  Security:	a27b0bb6-84fc-11ea-b5b4-641c67a117d8 (CVE-2020-8492)

Changes:
  head/lang/python-doc-html/distinfo
  head/lang/python36/Makefile
  head/lang/python36/Makefile.version
  head/lang/python36/distinfo
Comment 17 Danilo G. Baio freebsd_committer 2020-06-13 13:40:52 UTC
lang/python36 committed on ports r538669, waiting for MFH.
lang/python37 committed on ports r538670, waiting for MFH.
lang/python35 will wait a few more days, one of the patches were merged upstream and the other need changes.
Comment 18 commit-hook freebsd_committer 2020-06-13 14:08:53 UTC
A commit references this bug:

Author: dbaio
Date: Sat Jun 13 14:08:04 UTC 2020
New revision: 538674
URL: https://svnweb.freebsd.org/changeset/ports/538674

Log:
  security/vuxml: Update CVE-2019-18348 and CVE-2020-8492 entries

  Python 3.6 and 3.7 are not vulnerable in the ports tree anymore.
  Change range for python35 to <le>, suggested by swills.

  PR:		246984, 246738

Changes:
  head/security/vuxml/vuln.xml
Comment 19 commit-hook freebsd_committer 2020-06-15 11:18:07 UTC
A commit references this bug:

Author: dbaio
Date: Mon Jun 15 11:17:50 UTC 2020
New revision: 538871
URL: https://svnweb.freebsd.org/changeset/ports/538871

Log:
  MFH: r535638 r538669

  Update python38 doc to 3.8.3 after r535463

  lang/python36: Update to 3.6.10, Fix security issues

  The patches for CVE-2019-18348 and CVE-2020-8492 are in the 3.6 branch
  and will be present on the next release.

  Patch for applying CVE-2020-8492 fix here in the ports tree was reported
  and submitted by Mike Fisher <mfisher911@gmail.com> and
  Dani <i.dani@outlook.com>.

  PR:		246984
  Security:	ca595a25-91d8-11ea-b470-080027846a02 (CVE-2019-18348)
  Security:	a27b0bb6-84fc-11ea-b5b4-641c67a117d8 (CVE-2020-8492)

  Approved by:	ports-secteam (joneum)

Changes:
_U  branches/2020Q2/
  branches/2020Q2/lang/python-doc-html/distinfo
  branches/2020Q2/lang/python36/Makefile
  branches/2020Q2/lang/python36/Makefile.version
  branches/2020Q2/lang/python36/distinfo
Comment 20 commit-hook freebsd_committer 2020-06-20 14:22:06 UTC
A commit references this bug:

Author: dbaio
Date: Sat Jun 20 14:21:47 UTC 2020
New revision: 539739
URL: https://svnweb.freebsd.org/changeset/ports/539739

Log:
  lang/python35: Fix security issues

  The patches for CVE-2019-18348 and CVE-2020-8492 are in the 3.5 branch
  and will be present in a next release.

  PR:		246984
  Approved by:	python (with hat)
  MFH:		2020Q2
  Security:	ca595a25-91d8-11ea-b470-080027846a02 (CVE-2019-18348)
  Security:	a27b0bb6-84fc-11ea-b5b4-641c67a117d8 (CVE-2020-8492)

Changes:
  head/lang/python35/Makefile
  head/lang/python35/distinfo
Comment 21 Danilo G. Baio freebsd_committer 2020-06-20 14:31:27 UTC
(In reply to commit-hook from comment #20)

Patches for 3.5 were merged upstream and a new release `3.5.10 final: July 12, 2020` is expected.
I've updated the patchfiles id to the ones in the branch.
Waiting the MFH to close this PR.
Comment 22 commit-hook freebsd_committer 2020-06-22 11:05:21 UTC
A commit references this bug:

Author: dbaio
Date: Mon Jun 22 11:05:03 UTC 2020
New revision: 539801
URL: https://svnweb.freebsd.org/changeset/ports/539801

Log:
  MFH: r533797 r539739

  python 3.5 will reach End-of-life on 2020-09-13

  lang/python35: Fix security issues

  The patches for CVE-2019-18348 and CVE-2020-8492 are in the 3.5 branch
  and will be present in a next release.

  PR:		246984
  Approved by:	python (with hat)
  Security:	ca595a25-91d8-11ea-b470-080027846a02 (CVE-2019-18348)
  Security:	a27b0bb6-84fc-11ea-b5b4-641c67a117d8 (CVE-2020-8492)

  Approved by:	ports-secteam (blanket, backport of security fix)

Changes:
_U  branches/2020Q2/
  branches/2020Q2/lang/python35/Makefile
  branches/2020Q2/lang/python35/distinfo
Comment 23 Danilo G. Baio freebsd_committer 2020-06-22 11:07:25 UTC
All done, thank you all!
Comment 24 Kubilay Kocak freebsd_committer freebsd_triage 2020-06-22 11:31:19 UTC
Thank you Danilo!