Bug 256219

Summary: www/py-aiohttp: Update to 3.7.4.post0
Product: Ports & Packages Reporter: Dmitry Marakasov <amdmi3>
Component: Individual Port(s)Assignee: Dmitry Marakasov <amdmi3>
Status: Closed FIXED    
Severity: Affects Only Me CC: amdmi3, dbaio, jcfyecrayz, lwhsu, python, tech-lists, vishwin, wcarson.bugzilla, wen, ygy
Priority: --- Flags: koobs: maintainer-feedback? (vishwin)
koobs: maintainer-feedback? (amdmi3)
koobs: merge-quarterly?
Version: Latest   
Hardware: Any   
OS: Any   
URL: https://github.com/aio-libs/aiohttp/blob/v3.7.4.post0/CHANGES.rst
See Also: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=255025
Attachments:
Description Flags
Patch
none
Revised patch, update RUN_DEPENDS and python version require
koobs: maintainer-approval+
[patch] fix version ordering between 3.7.4 and 3.7.4.post0
none
[patch] fix version ordering between 3.7.4 and 3.7.4.post0 (v2) jcfyecrayz: maintainer-approval? (python)

Description Dmitry Marakasov freebsd_committer freebsd_triage 2021-05-28 17:13:23 UTC
Created attachment 225338 [details]
Patch
Comment 1 Kubilay Kocak freebsd_committer freebsd_triage 2021-05-30 02:03:46 UTC
Thanks Dmitry.

Any committer may commit to my ports for changes satisfying MAINTAINER_POLICY [1], with:

Approved by: koobs (implicit: MAINTAINER_POLICY)

[1] https://wiki.freebsd.org/KubilayKocak#MAINTAINER_POLICY
Comment 2 Dmitry Marakasov freebsd_committer freebsd_triage 2021-06-01 14:29:16 UTC
(In reply to Kubilay Kocak from comment #1)
This doesn't look practical. I'd rather wait for explicit approval, commit from your or timeout.
Comment 3 Kubilay Kocak freebsd_committer freebsd_triage 2021-06-01 23:40:43 UTC
(In reply to Dmitry Marakasov from comment #2)

Oh, what part(s) are not practical?
Comment 4 Kubilay Kocak freebsd_committer freebsd_triage 2021-06-01 23:46:42 UTC
Review items:

  - ports chardet is >= 4.0, *_DEPENDS stipulates <4.0, post0 bumped max version-spec to <5.0 [1]

  - Needs QA (poudriere, test suite) confirmation
  - Needs reverse dependents QA


[1] https://github.com/aio-libs/aiohttp/commit/934e5cbcc3ba8a952ff854c12b290ecdbb0856cb#diff-60f61ab7a8d1910d86d9fda2261620314edcae5894d5aaa236b821c7256badd7
Comment 5 Wen Heping freebsd_committer freebsd_triage 2021-06-02 03:01:42 UTC
Created attachment 225488 [details]
Revised patch, update RUN_DEPENDS and python version require

Revised patch, update RUN_DEPENDS and python version require
Comment 6 Dmitry Marakasov freebsd_committer freebsd_triage 2021-06-02 19:33:07 UTC
(In reply to Wen Heping from comment #5)
This works. Consumer ports have no related build failures.
Comment 7 Kubilay Kocak freebsd_committer freebsd_triage 2021-06-03 05:12:57 UTC
Comment on attachment 225488 [details]
Revised patch, update RUN_DEPENDS and python version require

Approved by: koobs (maintainer)
MFH: 2020Q2 (bugfix, security release(s))
Comment 8 Kubilay Kocak freebsd_committer freebsd_triage 2021-06-03 05:13:19 UTC
Approved to commit and merge
Comment 9 Guangyuan Yang freebsd_committer freebsd_triage 2021-06-03 10:57:49 UTC
*** Bug 254537 has been marked as a duplicate of this bug. ***
Comment 10 tech-lists 2021-06-03 11:16:44 UTC
This is still failing testport.

context:
poudriere-devel-3.3.99.20210521
amd64
stable/13-n245702-7ba858c624b: Mon May 24 17:51:56 BST 2021
% git -C /usr/ports rev-list --first-parent --count HEAD
547772

[...]

[00:00:09] ===========================================================================
[00:00:09] =>> Checking for filesystem violations... done
[00:00:09] =======================<phase: run-depends    >============================
[00:00:09] ===>   py38-aiohttp-3.6.2_1 depends on package: py38-attrs>=17.3.0 - not found
[00:00:09] ===>   Installing existing package /packages/All/py38-attrs-21.2.0.txz
[00:00:09] [pkg.zyxst.net] Installing py38-attrs-21.2.0...
[00:00:09] [pkg.zyxst.net] Extracting py38-attrs-21.2.0: .......... done
[00:00:09] ===>   py38-aiohttp-3.6.2_1 depends on package: py38-attrs>=17.3.0 - found
[00:00:09] ===>   Returning to build of py38-aiohttp-3.6.2_1
[00:00:09] ===>   py38-aiohttp-3.6.2_1 depends on package: py38-chardet>=2.0<4.0,1 - not found
[00:00:09] ===>   Installing existing package /packages/All/py38-chardet-4.0.0,1.txz
[00:00:09] [pkg.zyxst.net] Installing py38-chardet-4.0.0,1...
[00:00:09] [pkg.zyxst.net] Extracting py38-chardet-4.0.0,1: .......... done
[00:00:09] ===>   py38-aiohttp-3.6.2_1 depends on package: py38-chardet>=2.0<4.0,1 - not found
[00:00:09] *** Error code 1
[00:00:09] 
[00:00:09] Stop.
[00:00:09] make: stopped in /usr/ports/www/py-aiohttp
[00:00:09] Saving www/py-aiohttp | py38-aiohttp-3.6.2_1 wrkdir
[00:00:11] Saved www/py-aiohttp | py38-aiohttp-3.6.2_1 wrkdir to: /usr/local/poudriere/data/wrkdirs/13S-localhost-default/defaul
t/py38-aiohttp-3.6.2_1.txz
[00:00:13] build of www/py-aiohttp | py38-aiohttp-3.6.2_1 ended at Thu Jun  3 12:11:34 BST 2021
[00:00:13] build time: 00:00:09
[00:00:13] !!! build failure encountered !!!
[00:00:13] Error: Build failed in phase: run-depends
Comment 11 tech-lists 2021-06-03 11:18:34 UTC
...or has it not been applied to the ports tree yet?
Comment 12 tech-lists 2021-06-03 11:23:21 UTC
ignore my last two comments. Applied the patch and it works :D
Comment 13 commit-hook freebsd_committer freebsd_triage 2021-06-03 11:34:45 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/ports/commit/?id=ac412b55156cc77c8e96fb631a338a617749bcb7

commit ac412b55156cc77c8e96fb631a338a617749bcb7
Author:     Dmitry Marakasov <amdmi3@FreeBSD.org>
AuthorDate: 2021-06-03 11:15:22 +0000
Commit:     Dmitry Marakasov <amdmi3@FreeBSD.org>
CommitDate: 2021-06-03 11:26:51 +0000

    www/py-aiohttp: update to 3.7.4.post0

    PR:             256219
    Approved by:    koobs (maintainer)
    Security:       CVE-2021-21330
    Security:       3000acee-c45d-11eb-904f-14dae9d5a9d2
    MFH:            2020Q2 (bugfix, security release(s))

 www/py-aiohttp/Makefile                    | 16 +++++++---------
 www/py-aiohttp/distinfo                    |  6 +++---
 www/py-aiohttp/files/patch-setup.py (gone) | 27 ---------------------------
 3 files changed, 10 insertions(+), 39 deletions(-)
Comment 14 commit-hook freebsd_committer freebsd_triage 2021-06-03 11:37:47 UTC
A commit in branch 2021Q2 references this bug:

URL: https://cgit.FreeBSD.org/ports/commit/?id=618cb4a87e8f811e54889cd353f59847f8b55ba3

commit 618cb4a87e8f811e54889cd353f59847f8b55ba3
Author:     Dmitry Marakasov <amdmi3@FreeBSD.org>
AuthorDate: 2021-06-03 11:15:22 +0000
Commit:     Dmitry Marakasov <amdmi3@FreeBSD.org>
CommitDate: 2021-06-03 11:34:03 +0000

    www/py-aiohttp: update to 3.7.4.post0

    PR:             256219
    Approved by:    koobs (maintainer)
    Security:       CVE-2021-21330
    Security:       3000acee-c45d-11eb-904f-14dae9d5a9d2
    MFH:            2020Q2 (bugfix, security release(s))

    (cherry picked from commit ac412b55156cc77c8e96fb631a338a617749bcb7)

 www/py-aiohttp/Makefile                    | 10 +++++-----
 www/py-aiohttp/distinfo                    |  6 +++---
 www/py-aiohttp/files/patch-setup.py (gone) | 27 ---------------------------
 3 files changed, 8 insertions(+), 35 deletions(-)
Comment 15 Charlie Li freebsd_committer freebsd_triage 2021-06-03 13:35:39 UTC
Minor nitpick, but DISTVERSION=3.7.4.post0 results in a PORTVERSION of 3.7.4.p0, which is lower than 3.7.4.

$ pkg version -t 3.7.4 3.7.4.p0
>

I have something in my overlay to account for this, to show/derive PORTVERSION as 3.7.4post0 so that it is greater than 3.7.4.
Comment 16 Kubilay Kocak freebsd_committer freebsd_triage 2021-06-04 12:37:18 UTC
(In reply to Charlie Li from comment #15)

If this is an problem, please re-open the issue.

^Triage: Assign to committer that resolved
Comment 17 commit-hook freebsd_committer freebsd_triage 2021-06-06 19:03:36 UTC
A commit in branch 2021Q2 references this bug:

URL: https://cgit.FreeBSD.org/ports/commit/?id=25697c7f6769613885b3ed861f49bd42d65b0b24

commit 25697c7f6769613885b3ed861f49bd42d65b0b24
Author:     Dmitry Marakasov <amdmi3@FreeBSD.org>
AuthorDate: 2021-06-06 18:59:25 +0000
Commit:     Dmitry Marakasov <amdmi3@FreeBSD.org>
CommitDate: 2021-06-06 18:59:25 +0000

    www/py-aiohttp: update to 3.7.4.post0 (missed part)

    PR:             256219
    Approved by:    koobs (maintainer)
    Security:       CVE-2021-21330
    Security:       3000acee-c45d-11eb-904f-14dae9d5a9d2
    MFH:            2020Q2 (bugfix, security release(s))

    (cherry picked from commit ac412b55156cc77c8e96fb631a338a617749bcb7)

 www/py-aiohttp/Makefile | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)
Comment 18 John Hein 2021-06-06 19:57:16 UTC
Created attachment 225607 [details]
[patch] fix version ordering between 3.7.4 and 3.7.4.post0

(In reply to Kubilay Kocak from comment #16)
3.7.4.p0 < 3.7.4 is a problem in principal generally, but the problem is realized more concretely because of the vulnerability (for versions < 3.7.4):

================

===>  py37-aiohttp-3.7.4.p0 has known vulnerabilities:
py37-aiohttp-3.7.4.p0 is vulnerable:
  aiohttp -- open redirect vulnerability
  CVE: CVE-2021-21330
  WWW: https://vuxml.FreeBSD.org/freebsd/3000acee-c45d-11eb-904f-14dae9d5a9d2.html

1 problem(s) in 1 installed package(s) found.

================


The attached patch is perhaps more appropriate.  I cannot reopen this bug.  If necessary, we can open a new bug.


Potential commit message (feel free to edit):

Fix version for patch level update from 3.7.4 to 3.7.4.post0

3.7.4.post0 is a patch level after 3.7.4 was released.  And so the package version for 3.7.4.post0 should be considered a newer version than 3.7.4 (testable with 'pkg version -t 3.7.4 <newver>').  The pkg(8) version comparison rules treat 3.7.4.xxx as older than 3.7.4 (like an alpha, beta, or release candidate).

To fix that, specify that this patch level release is 3.7.4p0 which is considered newer than 3.7.4.  Use PORTVERSION to specify 3.7.4p0 that works for pkg(8) version ordering and DISTNAME to the actual distribution base filename.  The bsd.ports.mk conversion from DISTVERSION 3.7.4.post0 to PORTVERSION 3.7.4.p0 does not result in an appropriate ordering.


QA:
 portlint - ok
 testport - ok 


See also:

https://pypi.org/project/aiohttp/#history


https://docs.freebsd.org/en/books/porters-handbook/book.html#makefile-versions

  - notably "Example 5.5. Not Using DISTVERSION When the Version Contains Letters Meaning "Patch Level""
Comment 19 Charlie Li freebsd_committer freebsd_triage 2021-06-06 20:14:07 UTC
Reopening.

I think we should still have the full post0 in the PORTVERSION. Also, MASTER_SITES has to change slightly in order to fetch properly:

=> aiohttp-3.7.4.post0.tar.gz doesn't seem to exist in /distfiles/.
=> Attempting to fetch https://files.pythonhosted.org/packages/source/a/aiohttp-3.7.4.post0/aiohttp-3.7.4.post0.tar.gz
fetch: https://files.pythonhosted.org/packages/source/a/aiohttp-3.7.4.post0/aiohttp-3.7.4.post0.tar.gz: Not Found
=> Attempting to fetch https://pypi.org/packages/source/a/aiohttp-3.7.4.post0/aiohttp-3.7.4.post0.tar.gz
fetch: https://pypi.org/packages/source/a/aiohttp-3.7.4.post0/aiohttp-3.7.4.post0.tar.gz: Not Found
=> Attempting to fetch http://distcache.FreeBSD.org/ports-distfiles/aiohttp-3.7.4.post0.tar.gz
fetch: http://distcache.FreeBSD.org/ports-distfiles/aiohttp-3.7.4.post0.tar.gz: Not Found
=> Couldn't fetch it - please try to retrieve this
=> port manually into /distfiles/ and try again.
*** Error code 1

Here's what I have (which does fetch properly):
--- www/py-aiohttp/Makefile     2021-06-03 09:12:32.936243000 -0400
+++ www/py-aiohttp/Makefile     2021-06-03 09:33:48.207454000 -0400
@@ -1,10 +1,11 @@
 # Created by: Kubilay Kocak <koobs@FreeBSD.org>

 PORTNAME=      aiohttp
-DISTVERSION=   3.7.4.post0
+PORTVERSION=   3.7.4post0
 CATEGORIES=    www python
-MASTER_SITES=  CHEESESHOP
+MASTER_SITES=  CHEESESHOP/source/${PORTNAME:C/(.).*/\1/}/${PORTNAME}
 PKGNAMEPREFIX= ${PYTHON_PKGNAMEPREFIX}
+DISTNAME=      ${PORTNAME}-${PORTVERSION:S/post/.post/}

 MAINTAINER=    koobs@FreeBSD.org
 COMMENT=       Async http client/server framework (asyncio)
Comment 20 John Hein 2021-06-06 20:35:00 UTC
(In reply to Charlie Li from comment #19)
portlint complains about post0, but not about p0.  It could be appropriate to ignore portlint.
Comment 21 John Hein 2021-06-06 21:03:46 UTC
Created attachment 225610 [details]
[patch] fix version ordering between 3.7.4 and 3.7.4.post0 (v2)

Fix patch from comment 18 per Charlie Li's observation.  The previous patch rev was missing a change for MASTER_SITES.

I left "p0" because of the portlint whine.  It is up to the committer's discretion whether to ignore portlint and use "post0" in the pkg name instead (and reflect the upstream distribution name a bit more closely in the PKGNAME).

Next commit to a normal x.y.z should use DISTVERSION, remove DISTNAME and use the default CHEESESHOP for MASTER_SITES.

Note that upstream is using a pre-release and post-release numbering scheme that is the opposite of pkg(8) conventions (regarding whether it adds a final dot before that last part of the version "number"):

  pkg(8) - pre: 4.0.0.a0, post: 3.7.4p0 or 3.7.4post0

  upstream - pre: 4.0.0a0, post: 3.7.4.post0
Comment 22 John Hein 2021-06-06 21:05:35 UTC
(In reply to John Hein from comment #20)
Here's the portlint complaint:

FATAL: Makefile: PORTVERSION looks illegal. You should modify "3.7.4post0".
Comment 23 Kubilay Kocak freebsd_committer freebsd_triage 2021-06-07 05:27:44 UTC
(In reply to Charlie Li from comment #19)

If Dmitry can't get to this quickly, happy for you to take it
Comment 24 commit-hook freebsd_committer freebsd_triage 2021-06-23 10:01:04 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/ports/commit/?id=f3e4dbcb5ff2fe2a018f78f396a4247f1dd32cc9

commit f3e4dbcb5ff2fe2a018f78f396a4247f1dd32cc9
Author:     Li-Wen Hsu <lwhsu@FreeBSD.org>
AuthorDate: 2021-06-23 10:00:10 +0000
Commit:     Li-Wen Hsu <lwhsu@FreeBSD.org>
CommitDate: 2021-06-23 10:00:10 +0000

    security/vuxml: Fix version range of www/py-aiohttp

    This also marks 3.7.4.p0 as fixed.

    PR:     256219

 security/vuxml/vuln.xml | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
Comment 25 Li-Wen Hsu freebsd_committer freebsd_triage 2021-06-23 10:07:51 UTC
Let's just adjust the range of affected versions in vuxml. :)
Comment 26 John Hein 2021-06-25 15:22:01 UTC
(In reply to Li-Wen Hsu from comment #25)
Adjusting vuxml was fine, but it papers over the problem of the incorrect version specification.  However, since 3.7.4 never made it into the ports tree, the incorrect version currently committed doesn't hurt anything unless someone had a local change in their ports tree that had 3.7.4.

In the future, a "post" release like this should have a PKGNAME that is 3.7.4p0 (instead of 3.7.4.p0) so pkg version comparison (against 3.7.4) works correctly.

To avoid repo churn that would [correctly, but unnecessarily given that 3.7.4 was never committed] set the PKGNAME to 3.7.4p0, just re-closing this (after the vuxml change) without committing the version fix patch is fine.