Bug 172964 - [bsd.port.mk] USE_GITHUB feature not fully implemented
Summary: [bsd.port.mk] USE_GITHUB feature not fully implemented
Status: Closed Overcome By Events
Alias: None
Product: Documentation
Classification: Unclassified
Component: Books & Articles (show other bugs)
Version: Latest
Hardware: Any Any
: Normal Affects Only Me
Assignee: freebsd-ports-bugs (Nobody)
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2012-10-22 18:50 UTC by Shane
Modified: 2016-02-24 05:30 UTC (History)
0 users

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Shane 2012-10-22 18:50:01 UTC
As described in 5.4.7 of the porters handbook multiple distribution
files can be used in the form of 

  MASTER_SITES=  ftp://ftp.example1.com/:source1 \
   ftp://ftp.example2.com/:source2
  DISTFILES=  source1.tar.gz:source1 \
   source2.tar.gz:source2

I have just tried applying this to the new github settings and found
that it fails.

When multiple values are setup for GH_PROJECT and/or GH_TAGNAME the fetch
command gets a mangled download link with the group name instead of the
account/project/tag. I also tried with GH_ACCOUNT to match the groups
with no benefit.

eg while the correct link would be -
https://nodeload.github.com/sambler/myblender/tarball/sambler-2.64.2?dummy=/blender-sambler-2.64.tar.gz
having GH_PROJECT=  myblender:base turns the link into -
https://nodeload.github.com/baseblender-sambler-2.64.tar.gz

While I don't need to use a separate GH_ACCOUNT for what I am doing I
see that it could be required as well. The question there would be -
can project or tagname be setup with multiple sources or would all three
need to be setup for grouping? I could see a single account and tagname
being used with multiple project names to get the full sources needed,
but I doubt multiple tags with one project would be usable.

Looking ahead - this may require a way to define which extracted tarball
becomes WRKSRC. Maybe the DEFAULT group?

How-To-Repeat: The port I am working on isn't official but the files (without github
changes) are available at
http://redports.org/browser/sambler/graphics/blender-sambler

After removing MASTER_SITES DISTFILES FETCH_ARGS and WRKSRC I added -

USE_GITHUB= yes
GH_ACCOUNT= sambler
GH_PROJECT= myblender:base \
myblenderaddons:addons \
myblendercontrib:contrib \
myblendertranslations:trans
GH_TAGNAME= sambler-${PORTVERSION}.${PORTREVISION}:base \
addons-${PORTVERSION}.${PORTREVISION}:addons \
contrib-${PORTVERSION}.${PORTREVISION}:contrib \
translate-${PORTVERSION}.${PORTREVISION}:trans
Comment 1 Mark Linimon freebsd_committer freebsd_triage 2013-01-20 02:30:39 UTC
Responsible Changed
From-To: freebsd-ports-bugs->portmgr

Over to maintainer(s).
Comment 2 Shane 2013-01-21 02:04:30 UTC
As redports is still down you can get the port files I mention from
https://github.com/sambler/sambler-redports/tree/master/graphics/blender-sambler
Comment 3 Mark Linimon freebsd_committer freebsd_triage 2014-06-02 02:49:18 UTC
Infrastructure PR.
Comment 4 Mathieu Arnold freebsd_committer freebsd_triage 2015-06-12 20:31:56 UTC
I implemented this r387742.
Comment 5 Shane 2015-06-28 13:34:50 UTC
Not sure if I'm missing something but I can't get this to work right.

Testing with only the following in the Makefile -

PORTNAME=	blender-sambler
PORTVERSION=	2.74
PORTREVISION=	4
CATEGORIES=	graphics multimedia

USE_GITHUB=	yes
GH_ACCOUNT=	sambler:base
#		sambler:addons \
#		sambler:contrib \
#		sambler:trans
GH_PROJECT=	myblender:base
#		myblenderaddons:addons \
#		myblendercontrib:contrib \
#		myblendertranslations:trans
GH_TAGNAME=	sambler-${PORTVERSION}.${PORTREVISION}:base
#		addons-${PORTVERSION}.${PORTREVISION}:addons \
#		contrib-${PORTVERSION}.${PORTREVISION}:contrib \
#		translate-${PORTVERSION}.${PORTREVISION}:trans

.include <bsd.port.mk>

make fetch-urlall-list gives me two url's

## this url is wrong - it looks like GH/${PORTNAME}/${PORTNAME}/tar.gz/${PORTVERSION}
https://codeload.github.com/blender-sambler/blender-sambler/tar.gz/2.74?dummy=/blender-sambler-blender-sambler-2.74_GH0.tar.gz
http://distcache.FreeBSD.org/ports-distfiles/blender-sambler-blender-sambler-2.74_GH0.tar.gz

# this looks right - GH/${GH_ACCOUNT}/${GH_PROJECT}/tar.gz/${GH_TAGNAME}
https://codeload.github.com/sambler/myblender/tar.gz/sambler-2.74.4?dummy=/sambler-myblender-sambler-2.74.4_GH0.tar.gz
http://distcache.FreeBSD.org/ports-distfiles/sambler-myblender-sambler-2.74.4_GH0.tar.gz

If I remove the group name it will download the one item fine. Once I add a group name to just one item I get a duplicate download of the first item, with one of them being wrong. If I uncomment the other items I will get all four correct urls as well as the one extra for the base item.
Comment 6 Mathieu Arnold freebsd_committer freebsd_triage 2015-06-28 14:36:19 UTC
Remove :base, or use :DEFAULT.
Comment 7 Mathieu Arnold freebsd_committer freebsd_triage 2015-06-28 14:45:50 UTC
Is there some place where it tells you to use :base ? So that I can remove it with extreme prejudice...
Comment 8 Shane 2015-06-29 01:13:35 UTC
Yes having one item without a group name does fix this, thanks, but I will change the product to documentation for some improvement.

Looking through the docs... there isn't any github specific info yet, so I was using the DISTFILES info...

In the porters handbook in 5.4.7.1

paragraph 3 -
To support this, each entry in DISTFILES may be followed by a colon and a “tag name”. Each site listed in MASTER_SITES is then followed by a colon, and the tag that indicates which distribution files are downloaded from this site.

Example 5.4
MASTER_SITES=	ftp://ftp1.example.com/:source1 \
		http://www.example.com/:source2
DISTFILES=	source1.tar.gz:source1 \
		source2.tar.gz:source2

Example 5.5
MASTER_SITES=	ftp://ftp.example.com/:source1 \
		http://www.example.com/:source2
DISTFILES=	source1.tar.gz:source1 \
		source2.tar.gz:source2 \
		source3.tar.gz:source2

That is the most visible section and states "Each site listed in MASTER_SITES is then followed by a colon, and the tag..." which implies that each item gets a non-default group name (as I was trying). Later, example 5.7 does show items using no group and DEFAULT group, this is getting into more detailed info and could be skipped by many people.

Also 5.4.7.2 -> 3 says

Elements without a postfix are groupless, they all belong to the special group DEFAULT. Any elements postfixed with DEFAULT, is just being redundant unless an element belongs to both DEFAULT and other groups at the same time (check item 5).

These examples are equivalent but the first one is preferred:

MASTER_SITES=	alpha

MASTER_SITES=	alpha:DEFAULT

Followed by section 4 -

Groups are not exclusive, an element may belong to several different groups at the same time and a group can either have either several different elements or none at all.


The double "either" could be changed there.

I guess this is the closest part to what you refer, but I don't see mention of "don't add a group name to one element" or that one element "must be DEFAULT or nameless", in fact it states "a group can ... have ... none at all".

Before the GH_* group support was added I was using MASTER_SITES and DISTFILES setup with the same four groups I applied to the GH_* variables. That implies that the GH_* variables differ from the other variables supporting groups.

If you are adjusting the docs maybe you could also add some info for multiple parts to 5.4.2.3. USE_GITHUB. If requiring one DEFAULT group is unique to USE_GITHUB I think this would be the best place to explain that.

github variables could also be added to the variable list in 5.4.7.2 -> 7

I think that is all the relevant parts of the porters handbook.