Bug 220548

Summary: Mk/Scripts/cargo-crates.awk: "make cargo-crates" doesn't strip branch name
Product: Ports & Packages Reporter: Jan Beich <jbeich>
Component: Ports FrameworkAssignee: Tobias Kortkamp <tobik>
Status: Closed FIXED    
Severity: Affects Some People CC: portmgr, ports-bugs
Priority: --- Keywords: needs-qa
Version: Latest   
Hardware: Any   
OS: Any   
URL: https://reviews.freebsd.org/D11571
Attachments:
Description Flags
v0
none
v0.1
none
cargo-crates.diff none

Description Jan Beich freebsd_committer freebsd_triage 2017-07-07 21:41:52 UTC 
Created attachment 184161 [details]
v0

Not sure how much maintainers are supposed to edit "make cargo-crates" output but the following is probably not hard to fix.

https://github.com/servo/servo/blob/6e2e7151d8/Cargo.lock#L42 has

  [[package]]
  name = "angle"
  version = "0.2.0"
  source = "git+https://github.com/servo/angle?branch=servo#a1371e8a160128677af863d1d73f150862ba42b2"

which becomes

  GH_TUPLE=	servo:angle?branch=servo:a1371e8a160128677af863d1d73f150862ba42b2:angle \

and leads to

  $ make makesum
  => servo-angle?branch=servo-a1371e8a160128677af863d1d73f150862ba42b2_GH0.tar.gz doesn't seem to exist in /distfiles/.
  => Attempting to fetch https://codeload.github.com/servo/angle?branch=servo/tar.gz/a1371e8a160128677af863d1d73f150862ba42b2?dummy=/servo-angle?branch=servo-a1371e8a160128677af863d1d73f150862ba42b2_GH0.tar.gz
  fetch: https://codeload.github.com/servo/angle?branch=servo/tar.gz/a1371e8a160128677af863d1d73f150862ba42b2?dummy=/servo-angle?branch=servo-a1371e8a160128677af863d1d73f150862ba42b2_GH0.tar.gz: Bad Request
  => Attempting to fetch http://distcache.FreeBSD.org/ports-distfiles/servo-angle?branch=servo-a1371e8a160128677af863d1d73f150862ba42b2_GH0.tar.gz
  fetch: http://distcache.FreeBSD.org/ports-distfiles/servo-angle?branch=servo-a1371e8a160128677af863d1d73f150862ba42b2_GH0.tar.gz: Not Found
  => Couldn't fetch it - please try to retrieve this
  => port manually into /distfiles/ and try again.
  *** Error code 1

but fetching succeeds after ?branch=servo is removed.
Comment 1 Jan Beich freebsd_committer freebsd_triage 2017-07-07 21:58:42 UTC 
Created attachment 184162 [details]
v0.1

x11/alacritty has ?rev=<hash>, so let's tackle it as well.
Comment 2 Tobias Kortkamp freebsd_committer freebsd_triage 2017-07-08 08:11:00 UTC 
(In reply to Jan Beich from comment #1)
Looking at [1] I think we would need to strip ?tag= as well.

A dependency of e.g.

libc = {git = "https://github.com/rust-lang/libc", tag = "0.2.26"}

in Cargo.toml results in

[[package]]
name = "libc"
version = "0.2.26"
source = "git+https://github.com/rust-lang/libc?tag=0.2.26#288942e6858a4b2f8ee56338da5386263b9c4b82"

in Cargo.lock and then the wrong GH_TUPLE output again:

GH_TUPLE=	rust-lang:libc?tag=0.2.26:288942e6858a4b2f8ee56338da5386263b9c4b82:libc

which (given that we have an actual tag) probably should be

GH_TUPLE=	rust-lang:libc:0.2.26:libc

[1] https://github.com/rust-lang/cargo/blob/master/src/doc/specifying-dependencies.md#specifying-dependencies-from-git-repositories
Comment 3 Tobias Kortkamp freebsd_committer freebsd_triage 2017-07-08 08:39:10 UTC 
Created attachment 184169 [details]
cargo-crates.diff

Here's a patch that has special handling for ?tag=.
Comment 4 Tobias Kortkamp freebsd_committer freebsd_triage 2017-07-12 18:56:32 UTC 
I created a review on Phabricator with a slightly more sophisticated solution.
Comment 5 commit-hook freebsd_committer freebsd_triage 2017-07-19 14:00:52 UTC 
A commit references this bug:

Author: tobik
Date: Wed Jul 19 14:00:17 UTC 2017
New revision: 446198
URL: https://svnweb.freebsd.org/changeset/ports/446198

Log:
  When specifying Git dependencies in Cargo.toml developers can set 3
  additional keys: rev, branch, tag [1].  These are reflected in a
  projects' Cargo.lock file as e.g.

  git+https://github.com/servo/angle?branch=servo#a1371e8a160128677af863d1d73f150862ba42b2
  git+https://github.com/rust-lang/libc?tag=0.2.26#288942e6858a4b2f8ee56338da5386263b9c4b82

  Currently cargo-crates.awk generates the wrong output in these cases:

  GH_TUPLE=	servo:angle?branch=servo:a1371e8a160128677af863d1d73f150862ba42b2:angle \
  		rust-lang:libc?tag=0.2.26:288942e6858a4b2f8ee56338da5386263b9c4b82:libc

  Fix cargo-crates.awk to ignore the query string (except in the tag
  case) and generate

  GH_TUPLE=	servo:angle:a1371e8a160128677af863d1d73f150862ba42b2:angle \
  		rust-lang:libc:0.2.26:libc

  instead.

  [1] https://github.com/rust-lang/cargo/blob/master/src/doc/specifying-dependencies.md#specifying-dependencies-from-git-repositories

  PR:		220548
  Reported by:	jbeich
  Reviewed by:	jbeich, mat
  Differential Revision:	https://reviews.freebsd.org/D11571

Changes:
  head/Mk/Scripts/cargo-crates.awk