Bug 256581 - Mk/Uses/cargo.mk: CARGO_GIT_SUBDIR doesn't work with unpacked [dependencies.<crate>]
Summary: Mk/Uses/cargo.mk: CARGO_GIT_SUBDIR doesn't work with unpacked [dependencies.<...
Status: Open
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Ports Framework (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Some People
Assignee: FreeBSD Rust Team
URL:
Keywords: needs-patch
Depends on:
Blocks:
 
Reported: 2021-06-13 16:03 UTC by Jan Beich
Modified: 2021-09-09 01:29 UTC (History)
1 user (show)

See Also:
koobs: merge-quarterly?


Attachments
cargo-git-update.diff (1.30 KB, patch)
2021-06-16 17:57 UTC, Tobias Kortkamp
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Jan Beich freebsd_committer 2021-06-13 16:03:08 UTC
CARGO_GIT_SUBDIR expects each line to start with a crate name but fails to substitute local paths if a crate dependency spans multiple lines e.g.,

  [dependencies.wgt]
  package = "wgpu-types"
  git = "https://github.com/gfx-rs/wgpu"
  rev = "53eab747a32414232be45d47cae8a43a369395d0"

To reproduce:

1. Remove games/veloren/files/patch-wgpu
2. Build without network -> failed
3. Adjust CARGO_GIT_SUBDIR: s/wgc/wgpu-core/ + s/wgt/wgpu-types/
4. Build without network -> failed
Comment 1 Tobias Kortkamp freebsd_committer 2021-06-16 17:57:49 UTC
Created attachment 225862 [details]
cargo-git-update.diff

Ok, to get away from the sed hackery we need to let cargo do the
replacement with [patch] sections. This can maybe work like this:

1. Patch cargo to add an option to not force fetch Git sources when
   using cargo update (we abuse cargo update to trivially bump crate
   versions to avoid patching Cargo.lock ourselves; would be a shame to
   loose it); hack for that attached (against rust-1.52.0 branch in cargo 
   upstream).
2. Put appropriate [patch] sections into ${WRKDIR}/.cargo/config so
   that cargo can do the replacement itself. That currently requires
   the -Zpatch-in-config unstable feature [1].

   For dns/dog (much less complicated for experimentation than
   games/veloren) the [patch] section would look like:

GH_TUPLE=       llogiq:mutagen:c7abc956a10e8a3e2cc71f21279ea0a42f7b7c10:mutagen
CARGO_GIT_SUBDIR= mutagen:mutagen:mutagen       
=>
[patch.'https://github.com/llogiq/mutagen']
mutagen = { path = "${WRKSRC_mutagen}/mutagen" }

[1] https://github.com/rust-lang/cargo/issues/9269
Comment 2 Tobias Kortkamp freebsd_committer 2021-09-07 11:45:53 UTC
patch-in-config has been stabilized in Rust 1.56 (2021-10-21).

To get slightly more concrete it might look like this:

diff --git a/Mk/Uses/cargo.mk b/Mk/Uses/cargo.mk
index 3d065a45ed3b..46a2ffd40d93 100644
--- a/Mk/Uses/cargo.mk
+++ b/Mk/Uses/cargo.mk
@@ -107,8 +107,8 @@ STRIP_CMD=  ${LOCALBASE}/bin/strip # unsupported e_type with base strip

 # Helper to shorten cargo calls.
 CARGO_CARGO_RUN= \
-       cd ${WRKSRC} && ${SETENV} ${MAKE_ENV} ${CARGO_ENV} \
-               ${CARGO_CARGO_BIN}
+       cd ${WRKSRC} && ${SETENV} ${MAKE_ENV} ${CARGO_ENV} RUSTC_BOOTSTRAP=1 CARGO_FREEBSD_PORTS_SKIP_GIT_UPDATE=1 \
+               ${CARGO_CARGO_BIN} -Zpatch-in-config

 # User arguments for cargo targets.
 CARGO_BUILD_ARGS?=
@@ -296,6 +296,8 @@ cargo-configure:
        @${ECHO_CMD} "directory = '${CARGO_VENDOR_DIR}'" >> ${WRKDIR}/.cargo/config
        @${ECHO_CMD} "[source.crates-io]" >> ${WRKDIR}/.cargo/config
        @${ECHO_CMD} "replace-with = 'cargo'" >> ${WRKDIR}/.cargo/config
+       @${ECHO_CMD} "[patch.'https://github.com/llogiq/mutagen']" >> ${WRKDIR}/.cargo/config
+       @${ECHO_CMD} "mutagen = { path = '${WRKSRC_mutagen}/mutagen' }" >> ${WRKDIR}/.cargo/config
        @if ! ${GREP} -qF '[profile.release]' ${CARGO_CARGOTOML}; then \
                ${ECHO_CMD} "" >> ${CARGO_CARGOTOML}; \
                ${ECHO_CMD} "[profile.release]" >> ${CARGO_CARGOTOML}; \
diff --git a/dns/dog/Makefile b/dns/dog/Makefile
index e7b9aa7039de..2cb66d1895fa 100644
--- a/dns/dog/Makefile
+++ b/dns/dog/Makefile
@@ -90,8 +90,15 @@ CARGO_CRATES=        addr2line-0.14.0 \
                winapi-i686-pc-windows-gnu-0.4.0 \
                winapi-x86_64-pc-windows-gnu-0.4.0 \
                winreg-0.6.2
-CARGO_USE_GITHUB=      yes
-CARGO_GIT_SUBDIR=      mutagen:mutagen:mutagen
+# [[package]]
+# name = "mutagen"
+# source = "git+https://github.com/llogiq/mutagen#c7abc956a10e8a3e2cc71f21279ea0a42f7b7c10"
+#
+# make cargo-crates to turn entries like the above into
+#
+# CARGO_CRATES+=       git+https://github.com/llogiq/mutagen\#c7abc956a10e8a3e2cc71f21279ea0a42f7b7c10
+#
+# which cargo.mk can then consume to create the appropriate [patch] sections and GH_TUPLE.

 PORTDOCS=      README.md
Comment 3 Tobias Kortkamp freebsd_committer 2021-09-08 19:32:26 UTC
(In reply to Tobias Kortkamp from comment #2)
Working code for that is here:
https://github.com/t6/freebsd-ports/compare/main...t6:tobik/cargo-git-patch-in-config