Bug 247666 - games/anki: Update to new build system
Summary: games/anki: Update to new build system
Status: Open
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Some People
Assignee: Kai Knoblich
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2020-06-30 19:00 UTC by David Schlachter
Modified: 2020-08-14 22:59 UTC (History)
1 user (show)

See Also:
kai: maintainer-feedback+


Attachments
maturin.patch (42.97 KB, patch)
2020-07-05 09:28 UTC, Kai Knoblich
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description David Schlachter 2020-06-30 19:00:56 UTC
As noted in the ports commit history, Anki uses a new set of build scripts since version 2.1.17 (Jan 2020). I opened this bug report to organize notes about updating the port to the new build system.

Changes necessary to build:

- set USE_GITHUB to get latest source release
- use gmake instead of make
- bash required to build
- change all hardcoded paths to bash in the build scripts from /bin/bash to /usr/local/bin/bash (about 15 occurrences)
- set the PYTHON_BIN make environment variable to the actual python binary's name (rather than default "python3" in Makefile) (can be set with ports variable PYTHON_CMD?)

Changes that might be needed:

- build script attempts to fetch a pyqt5 5.13.2 with pip, which seemed to fail. Maybe add as a dependency and install with ports instead?
- build script attempts to install maturin with pip, which fails when building native module with rust ('building wheel'). However, continuous integration for maturin shows successful FreeBSD builds:
  https://cirrus-ci.com/github/PyO3/maturin
  https://github.com/PyO3/maturin/blob/master/.cirrus.yml
I tried building maturin according to the steps in cirrus.yml and it built fine. Maybe we need a maturin port?

Other possible changes that might be needed (need to investigate):

- prefix isn't specified in build script anymore, not sure how this is handled.
Comment 1 Kai Knoblich freebsd_committer 2020-07-05 09:21:26 UTC
(In reply to David Schlachter from comment #0)

Thank you for the notes. I add my findings/notes that have accumulated over the past few months.  Here's a rough overview of the build process (Anki 2.1.26): 

The build process consists of four main steps where the order is important:

1. Build the Anki Rust library in ${WRKSRC}/rslib 
        - Bootstrap/Setup Rust via rustup because it's bound to a nightly release (= 2020-02-27)
        - Create a ".so" file that is used by step 2

2. Build Rust/Python bridge in ${WRKSRC}/rspy
        - Bootstrap/Setup Rust via rustup because it's bound to a nightly release (= 2020-02-27)
        - Use maturin (yet to be ported) to create a Python compatible ".so" file (packaged in a Python Wheel)
        - For Python 3.7 the ".so" would be ankirspy.cpython-37m.so

3. Build/Install the Python library code in ${WRKSRC}/pylib to ${PYTHONPREFIX_SITELIBDIR}
        - The setup.py has a reference to "ankirspy" (created in step 2):

> install_requires.append("ankirspy==2.1.26")

        - The package is also available via PyPI -> anki 

4. Build/Install the Python Qt code in ${WRKSRC}/qt to ${PYTHONPREFIX_SITELIBDIR}
        - The package is also available via PyPI -> aqt

Before the build begins, some preparations are made. Most of them opens a network connection hence they must be done before the whole build process, e.g. via the "post-extract" and "patch" targets:

- Echo the actual build hash to ${WRKSRC}/meta/buildhash
        - Buildhash needs to be determined beforehand via "git rev-parse --short=8 HEAD"
        - Alternatively the string "dev" can be used as a build hash

- Fetch latest translation files (via Git) and place them to
        - ${WRKSRC}/rslib/ftl/repo
        - ${WRKSRC}/qt/ftl/repo
        - ${WRKSRC}/qt/po/repo

- Bootstrap Rust via "rustup" (downloads and setups a pre-compiled Rust environment)

- Determine/Sync/Download the Cargo crates
        - Both ${WRKSRC}/rslib and ${WRKSRC}/rspy directories have no Cargo.lock files
        - Cargo.toml needs to be patched -> remove vendored OpenSSL: 

> [target.'cfg(linux)'.dependencies]
> reqwest = { version = "0.10.1", features = ["json", "socks", "stream", "native-tls-vendored"] }

- Install the PyQt5 dependencies (if required for build).

The biggest challenge is at the moment to get a successful compilation of the Rust code as it works only with a specific rust-nightly release that is bootstrapped/setup via "rustup".

Here's a short overview:

>                                 | Anki 2.1.26           | HEAD branch (Commit 04b1ca75)
> --------------------------------+-----------------------+------------------------------
> rust 1.44.1_1                   | fails                 | Partially OK [1]
> --------------------------------+-----------------------+------------------------------
> rust-nightly 1.46.0.20200622    | fails                 | Partially OK [2]
> --------------------------------+-----------------------+------------------------------
> rust-nightly 1.43.0.20200228    | Partially OK [2]      | not tested
> --------------------------------+-----------------------+------------------------------
> rustup                          | OK                    | OK
> --------------------------------+-----------------------+------------------------------

[1] rslib builds fine but rspy fails because it uses a "pyo3" crate which can only be compiled with rust-nightly at the moment
[2] rslib builds fine but rspy fails because the rust-nightly date cannot be determined (maybe this can be patched out).
Comment 2 Kai Knoblich freebsd_committer 2020-07-05 09:28:52 UTC
Created attachment 216211 [details]
maturin.patch

(In reply to David Schlachter from comment #0)

Attached is a draft of devel/maturin that should build fine and can be used to build the Rust/Python library.
Comment 3 PauAmma 2020-08-14 22:59:02 UTC
Anki has lately taken to nagging me about 2.1.30 on startup.