MASTER_SITE_SUBDIRS in bsd.sites.mk has the following substitution for CHEESESHOP: CHEESESHOP:source/${DISTNAME:C/(.).*/\1/}/${DISTNAME:C/(.*)-[0-9].*/\1/} This breaks databases/pyspatialite, which sets DISTNAME to "${PORTNAME}-${PORTVERSION}-alpha-0". Since the regular expressions are greedy, the last CHEESESHOP substitution matches "pyspatialite-3.0.1-alpha" instead of "pyspatialite". Perhaps PORTNAME should be used instead of DISTNAME?
pyspatialite is an exception, and using PORTNAME instead of DISTNAME breaks more ports that it fixes You can use this workaround: MASTER_SITES= CHEESESHOP/source/p/${PORTNAME}
A commit references this bug: Author: sunpoet Date: Sun Jan 3 17:50:44 UTC 2016 New revision: 405177 URL: https://svnweb.freebsd.org/changeset/ports/405177 Log: - Update MASTER_SITES and unbreak this port [1] - While I'm here, use DISTVERSIONSUFFIX instead of DISTNAME PR: 205762 Submitted by: rakuco Obtained from: antoine [1] Approved by: portmgr (blanket) With hat: python Changes: head/databases/pyspatialite/Makefile
I think this should be fixed ASAP. But we also need to improve CHEESESHOP substitution. It does not recognize DISTVERSIONPREFIX and DISTVERSIONSUFFIX which are usually non-digit (if set). It might work if we replace [0-9].* with DISTVERSIONFULL. At least it works for this port and x11-fonts/py-bdflib.
Something like this? - CHEESESHOP:source/${DISTNAME:C/(.).*/\1/}/${DISTNAME:C/(.*)-[0-9].*/\1/} \ + CHEESESHOP:source/${DISTNAME:C/(.).*/\1/}/${DISTNAME:S/-${DISTVERSIONFULL}$//} \ This should be tested with empty DISTDIR / disabled MASTER_SITE_BACKUP
I have the following errors with the patch applied: fetch: https://pypi.python.org/packages/source/a/anyconfig/anyconfig-0.0.5.tar.gz: Not Found fetch: https://pypi.python.org/packages/source/c/cycler/cycler-0.9.0.tar.gz: Not Found fetch: https://pypi.python.org/packages/source/m/mongokit/mongokit-0.8.3.tar.gz: size mismatch: expected 68024, actual 68015 fetch: https://pypi.python.org/packages/source/S/SimPy/SimPy-2.3.1.tar.gz: Not Found fetch: https://pypi.python.org/packages/source/u/unittestplus/unittestplus-1.1.1.zip: Not Found fetch: https://pypi.python.org/packages/source/P/PyX/PyX-0.12.1.tar.gz: Not Found fetch: https://pypi.python.org/packages/source/B/BitVector/BitVector-3.1.tar.gz: Not Found fetch: https://pypi.python.org/packages/source/p/python-novaclient/python-novaclient-2.6.8.tar.gz: Not Found fetch: https://pypi.python.org/packages/source/d/diff-match-patch-20121119/diff-match-patch-20121119.tar.gz: Not Found fetch: https://pypi.python.org/packages/source/b/blogofile/blogofile-0.7.1.tar.gz: Not Found fetch: https://pypi.python.org/packages/source/d/django-cms/django-cms-2.4.1.tar.gz: size mismatch: expected 4436793, actual 4650951 fetch: https://pypi.python.org/packages/source/T/TraitsBackendWX/TraitsBackendWX-3.2.0.tar.gz: Not Found fetch: https://pypi.python.org/packages/source/T/TraitsGUI/TraitsGUI-3.1.0.tar.gz: Not Found Note: those errors may happen without the patch applied..
It looks like all errors except for textproc/py-diff-match-patch are unrelated to the patch and the ports have been broken for a while. It's worth considering the problems in devel/py-cycler and www/py-blogofile though: basically the tarballs and the directories differ in casing, which can also cause problems. Not sure if the only solution to those is to set MASTER_SITE_SUBDIR manually. (In reply to Antoine Brodin from comment #5) > I have the following errors with the patch applied: > > fetch: https://pypi.python.org/packages/source/a/anyconfig/anyconfig-0.0.5.tar.gz: Not Found Fixed in ports r405193. > fetch: https://pypi.python.org/packages/source/c/cycler/cycler-0.9.0.tar.gz: Not Found This was broken in ports r404408, I've replied to the commit email. > fetch: https://pypi.python.org/packages/source/m/mongokit/mongokit-0.8.3.tar.gz: size mismatch: expected 68024, actual 68015 Fixed in ports r405195. > fetch: https://pypi.python.org/packages/source/S/SimPy/SimPy-2.3.1.tar.gz: Not Found Fixed in ports r405196 and ports r405227. > fetch: https://pypi.python.org/packages/source/u/unittestplus/unittestplus-1.1.1.zip: Not Found The project has vanished from PyPI and I couldn't find an official website anymore. I'm tempted to deprecate it and mark the port for removal. > fetch: https://pypi.python.org/packages/source/P/PyX/PyX-0.12.1.tar.gz: Not Found This was fixed a few weeks ago in ports r403796. > fetch: https://pypi.python.org/packages/source/B/BitVector/BitVector-3.1.tar.gz: Not Found Fixed in ports r405198. > fetch: https://pypi.python.org/packages/source/p/python-novaclient/python-novaclient-2.6.8.tar.gz: Not Found I couldn't find this release/tag in PyPI, GitHub or OpenStack's git repositories. > fetch: https://pypi.python.org/packages/source/d/diff-match-patch-20121119/diff-match-patch-20121119.tar.gz: Not Found This one looks like the only legitimate error: the tarballs' parent director should be diff-match-patch, not diff-match-patch-20121119. > fetch: https://pypi.python.org/packages/source/b/blogofile/blogofile-0.7.1.tar.gz: Not Found This has the same problem as devel/py-cycler: the tarball is called "blogofile-0.7.1.tar.gz" (lowercase 'b'), whereas the PyPI project is called "Blogofile" (uppercase 'B'), which is also the tarball's parent directory name. > fetch: https://pypi.python.org/packages/source/d/django-cms/django-cms-2.4.1.tar.gz: size mismatch: expected 4436793, actual 4650951 The new tarball has quite a few differences, so I left it broken. > fetch: https://pypi.python.org/packages/source/T/TraitsBackendWX/TraitsBackendWX-3.2.0.tar.gz: Not Found I couldn't find the tarball anywhere. > fetch: https://pypi.python.org/packages/source/T/TraitsGUI/TraitsGUI-3.1.0.tar.gz: Not Found Fixed in ports r405199.
Assign to committer that's resolving/committing
The ports I mentioned in comment #6 have all either been fixed or marked BROKEN so they shouldn't interfere with your tests.
Re-open, merge-quarterly flag is pending.
We don't want this in quarterly as this requires lots of MFH. But I don't think it's fixed in head?
databases/pyspatialite itself was fixed with the commit in comment #2, it's not clear if there's anything we want to change in the CHEESESHOP substitution though. Was the change proposed in comment #4 tested after comment #8?
Created attachment 174364 [details] patch for bsd.sites.mk
Ports failing to fetch with the patch applied: => Attempting to fetch https://pypi.python.org/packages/source/P/PyX/PyX-0.12.1.tar.gz fetch: https://pypi.python.org/packages/source/P/PyX/PyX-0.12.1.tar.gz: Not Found => Attempting to fetch https://files.pythonhosted.org/packages/source/P/PyX/PyX-0.12.1.tar.gz fetch: https://files.pythonhosted.org/packages/source/P/PyX/PyX-0.12.1.tar.gz: Not Found => Couldn't fetch it - please try to retrieve this => port manually into /tmp/distfiles/ and try again. *** Error code 1 Stop. make: stopped in /usr/ports/graphics/py-PyX12 -> it fails to fetch without the patch applied too => Attempting to fetch https://pypi.python.org/packages/source/P/PyPrind-2.9.8/PyPrind-2.9.8.tar.gz fetch: https://pypi.python.org/packages/source/P/PyPrind-2.9.8/PyPrind-2.9.8.tar.gz: Not Found => Attempting to fetch https://files.pythonhosted.org/packages/source/P/PyPrind-2.9.8/PyPrind-2.9.8.tar.gz fetch: https://files.pythonhosted.org/packages/source/P/PyPrind-2.9.8/PyPrind-2.9.8.tar.gz: Not Found => Couldn't fetch it - please try to retrieve this => port manually into /tmp/distfiles/ and try again. *** Error code 1 Stop. make: stopped in /usr/ports/misc/py-pyprind -> this port has a bogus DISTVERSIONPREFIX => Attempting to fetch https://pypi.python.org/packages/source/h/haproxy_log_analysis-2.0b0/haproxy_log_analysis-2.0b0.tar.gz fetch: https://pypi.python.org/packages/source/h/haproxy_log_analysis-2.0b0/haproxy_log_analysis-2.0b0.tar.gz: Not Found => Attempting to fetch https://files.pythonhosted.org/packages/source/h/haproxy_log_analysis-2.0b0/haproxy_log_analysis-2.0b0.tar.gz fetch: https://files.pythonhosted.org/packages/source/h/haproxy_log_analysis-2.0b0/haproxy_log_analysis-2.0b0.tar.gz: Not Found => Couldn't fetch it - please try to retrieve this => port manually into /tmp/distfiles/ and try again. *** Error code 1 Stop. make: stopped in /usr/ports/net/py-haproxy-log-analysis -> this port should use DISTVERSION => Attempting to fetch http://files.turpial.org.ve/sources/stable/libturpial-1.7.0.tar.gz fetch: http://files.turpial.org.ve/sources/stable/libturpial-1.7.0.tar.gz: Bad Gateway => Attempting to fetch https://pypi.python.org/packages/source/l/libturpial/libturpial-1.7.0.tar.gz fetch: https://pypi.python.org/packages/source/l/libturpial/libturpial-1.7.0.tar.gz: Not Found => Attempting to fetch https://files.pythonhosted.org/packages/source/l/libturpial/libturpial-1.7.0.tar.gz fetch: https://files.pythonhosted.org/packages/source/l/libturpial/libturpial-1.7.0.tar.gz: Not Found => Couldn't fetch it - please try to retrieve this => port manually into /tmp/distfiles/ and try again. *** Error code 1 Stop. make: stopped in /usr/ports/net-im/py-libturpial -> it fails to fetch without the patch applied too => Attempting to fetch http://www.c-s.li/ports/stem-1.4.1.tar.bz2 fetch: http://www.c-s.li/ports/stem-1.4.1.tar.bz2: size unknown fetch: http://www.c-s.li/ports/stem-1.4.1.tar.bz2: size of remote file is not known stem-1.4.1.tar.bz2 0 B 0 Bpsstem-1.4.1.tar.bz2 13 kB 196 MBps 00m00s => Fetched file size mismatch (expected 1570151, actual 14020) => Trying next site => Attempting to fetch https://pypi.python.org/packages/source/s/stem/stem-1.4.1.tar.bz2 fetch: https://pypi.python.org/packages/source/s/stem/stem-1.4.1.tar.bz2: Not Found => Attempting to fetch https://files.pythonhosted.org/packages/source/s/stem/stem-1.4.1.tar.bz2 fetch: https://files.pythonhosted.org/packages/source/s/stem/stem-1.4.1.tar.bz2: Not Found => Couldn't fetch it - please try to retrieve this => port manually into /tmp/distfiles/ and try again. *** Error code 1 Stop. make: stopped in /usr/ports/security/py-stem -> it fails to fetch without the patch applied too => Attempting to fetch https://pypi.python.org/packages/source/p/pycryptopp-0.7.1.869544967005693312591928092448767568728501330214/pycryptopp-0.7.1.869544967005693312591928092448767568728501330214.tar.gz fetch: https://pypi.python.org/packages/source/p/pycryptopp-0.7.1.869544967005693312591928092448767568728501330214/pycryptopp-0.7.1.869544967005693312591928092448767568728501330214.tar.gz: Not Found => Attempting to fetch https://files.pythonhosted.org/packages/source/p/pycryptopp-0.7.1.869544967005693312591928092448767568728501330214/pycryptopp-0.7.1.869544967005693312591928092448767568728501330214.tar.gz fetch: https://files.pythonhosted.org/packages/source/p/pycryptopp-0.7.1.869544967005693312591928092448767568728501330214/pycryptopp-0.7.1.869544967005693312591928092448767568728501330214.tar.gz: Not Found => Couldn't fetch it - please try to retrieve this => port manually into /tmp/distfiles/ and try again. *** Error code 1 Stop. make: stopped in /usr/ports/security/py-pycryptopp -> abuses PORTVERSION/DISTNAME, should be fixed => Attempting to fetch https://pypi.python.org/packages/source/d/diff-match-patch-20121119/diff-match-patch-20121119.tar.gz fetch: https://pypi.python.org/packages/source/d/diff-match-patch-20121119/diff-match-patch-20121119.tar.gz: Not Found => Attempting to fetch https://files.pythonhosted.org/packages/source/d/diff-match-patch-20121119/diff-match-patch-20121119.tar.gz fetch: https://files.pythonhosted.org/packages/source/d/diff-match-patch-20121119/diff-match-patch-20121119.tar.gz: Not Found => Couldn't fetch it - please try to retrieve this => port manually into /tmp/distfiles/ and try again. *** Error code 1 Stop. make: stopped in /usr/ports/textproc/py-diff-match-patch -> abuses PORTVERSION, should be fixed => Attempting to fetch https://pypi.python.org/packages/source/d/django-cms/django-cms-2.4.1.tar.gz fetch: https://pypi.python.org/packages/source/d/django-cms/django-cms-2.4.1.tar.gz: size mismatch: expected 4436793, actual 4650951 => Attempting to fetch https://files.pythonhosted.org/packages/source/d/django-cms/django-cms-2.4.1.tar.gz fetch: https://files.pythonhosted.org/packages/source/d/django-cms/django-cms-2.4.1.tar.gz: size mismatch: expected 4436793, actual 4650951 => Couldn't fetch it - please try to retrieve this => port manually into /tmp/distfiles/ and try again. *** Error code 1 Stop. make: stopped in /usr/ports/www/py-django-cms -> it fails to fetch without the patch applied too, rerolled? => Attempting to fetch https://pypi.python.org/packages/source/T/TraitsBackendWX/TraitsBackendWX-3.2.0.tar.gz fetch: https://pypi.python.org/packages/source/T/TraitsBackendWX/TraitsBackendWX-3.2.0.tar.gz: Not Found => Attempting to fetch https://files.pythonhosted.org/packages/source/T/TraitsBackendWX/TraitsBackendWX-3.2.0.tar.gz fetch: https://files.pythonhosted.org/packages/source/T/TraitsBackendWX/TraitsBackendWX-3.2.0.tar.gz: Not Found => Couldn't fetch it - please try to retrieve this => port manually into /tmp/distfiles/ and try again. *** Error code 1 Stop. make: stopped in /usr/ports/x11-toolkits/py-traitsbackendwx -> it fails to fetch without the patch applied too, already marked BROKEN
A commit references this bug: Author: antoine Date: Sun Sep 4 09:54:56 UTC 2016 New revision: 421318 URL: https://svnweb.freebsd.org/changeset/ports/421318 Log: Remove bogus DISTVERSIONPREFIX PR: 205762 Changes: head/misc/py-pyprind/Makefile
A commit references this bug: Author: antoine Date: Sun Sep 4 09:58:40 UTC 2016 New revision: 421319 URL: https://svnweb.freebsd.org/changeset/ports/421319 Log: Use DISTVERSION PR: 205762 Changes: head/net/py-haproxy-log-analysis/Makefile
A commit references this bug: Author: antoine Date: Sun Sep 4 10:03:20 UTC 2016 New revision: 421320 URL: https://svnweb.freebsd.org/changeset/ports/421320 Log: Use the same PORTVERSION as the one on Pypi PR: 205762 Changes: head/security/py-pycryptopp/Makefile
A commit references this bug: Author: antoine Date: Sun Sep 4 10:05:46 UTC 2016 New revision: 421321 URL: https://svnweb.freebsd.org/changeset/ports/421321 Log: Use the same PORTVERSION as the one on Pypi PR: 205762 Changes: head/textproc/py-diff-match-patch/Makefile head/textproc/py-diff-match-patch/distinfo
A commit references this bug: Author: antoine Date: Sun Sep 4 10:23:14 UTC 2016 New revision: 421322 URL: https://svnweb.freebsd.org/changeset/ports/421322 Log: Improve CHEESESHOP subdir substitution, it didn't work well with DISTVERSION containing non digits. PR: 205762 With hat: portmgr Changes: head/Mk/bsd.sites.mk