As noted in the ports commit history, Anki uses a new set of build scripts since version 2.1.17 (Jan 2020). I opened this bug report to organize notes about updating the port to the new build system. Changes necessary to build: - set USE_GITHUB to get latest source release - use gmake instead of make - bash required to build - change all hardcoded paths to bash in the build scripts from /bin/bash to /usr/local/bin/bash (about 15 occurrences) - set the PYTHON_BIN make environment variable to the actual python binary's name (rather than default "python3" in Makefile) (can be set with ports variable PYTHON_CMD?) Changes that might be needed: - build script attempts to fetch a pyqt5 5.13.2 with pip, which seemed to fail. Maybe add as a dependency and install with ports instead? - build script attempts to install maturin with pip, which fails when building native module with rust ('building wheel'). However, continuous integration for maturin shows successful FreeBSD builds: https://cirrus-ci.com/github/PyO3/maturin https://github.com/PyO3/maturin/blob/master/.cirrus.yml I tried building maturin according to the steps in cirrus.yml and it built fine. Maybe we need a maturin port? Other possible changes that might be needed (need to investigate): - prefix isn't specified in build script anymore, not sure how this is handled.
(In reply to David Schlachter from comment #0) Thank you for the notes. I add my findings/notes that have accumulated over the past few months. Here's a rough overview of the build process (Anki 2.1.26): The build process consists of four main steps where the order is important: 1. Build the Anki Rust library in ${WRKSRC}/rslib - Bootstrap/Setup Rust via rustup because it's bound to a nightly release (= 2020-02-27) - Create a ".so" file that is used by step 2 2. Build Rust/Python bridge in ${WRKSRC}/rspy - Bootstrap/Setup Rust via rustup because it's bound to a nightly release (= 2020-02-27) - Use maturin (yet to be ported) to create a Python compatible ".so" file (packaged in a Python Wheel) - For Python 3.7 the ".so" would be ankirspy.cpython-37m.so 3. Build/Install the Python library code in ${WRKSRC}/pylib to ${PYTHONPREFIX_SITELIBDIR} - The setup.py has a reference to "ankirspy" (created in step 2): > install_requires.append("ankirspy==2.1.26") - The package is also available via PyPI -> anki 4. Build/Install the Python Qt code in ${WRKSRC}/qt to ${PYTHONPREFIX_SITELIBDIR} - The package is also available via PyPI -> aqt Before the build begins, some preparations are made. Most of them opens a network connection hence they must be done before the whole build process, e.g. via the "post-extract" and "patch" targets: - Echo the actual build hash to ${WRKSRC}/meta/buildhash - Buildhash needs to be determined beforehand via "git rev-parse --short=8 HEAD" - Alternatively the string "dev" can be used as a build hash - Fetch latest translation files (via Git) and place them to - ${WRKSRC}/rslib/ftl/repo - ${WRKSRC}/qt/ftl/repo - ${WRKSRC}/qt/po/repo - Bootstrap Rust via "rustup" (downloads and setups a pre-compiled Rust environment) - Determine/Sync/Download the Cargo crates - Both ${WRKSRC}/rslib and ${WRKSRC}/rspy directories have no Cargo.lock files - Cargo.toml needs to be patched -> remove vendored OpenSSL: > [target.'cfg(linux)'.dependencies] > reqwest = { version = "0.10.1", features = ["json", "socks", "stream", "native-tls-vendored"] } - Install the PyQt5 dependencies (if required for build). The biggest challenge is at the moment to get a successful compilation of the Rust code as it works only with a specific rust-nightly release that is bootstrapped/setup via "rustup". Here's a short overview: > | Anki 2.1.26 | HEAD branch (Commit 04b1ca75) > --------------------------------+-----------------------+------------------------------ > rust 1.44.1_1 | fails | Partially OK [1] > --------------------------------+-----------------------+------------------------------ > rust-nightly 1.46.0.20200622 | fails | Partially OK [2] > --------------------------------+-----------------------+------------------------------ > rust-nightly 1.43.0.20200228 | Partially OK [2] | not tested > --------------------------------+-----------------------+------------------------------ > rustup | OK | OK > --------------------------------+-----------------------+------------------------------ [1] rslib builds fine but rspy fails because it uses a "pyo3" crate which can only be compiled with rust-nightly at the moment [2] rslib builds fine but rspy fails because the rust-nightly date cannot be determined (maybe this can be patched out).
Created attachment 216211 [details] maturin.patch (In reply to David Schlachter from comment #0) Attached is a draft of devel/maturin that should build fine and can be used to build the Rust/Python library.
Anki has lately taken to nagging me about 2.1.30 on startup.
MARKED AS SPAM
Created attachment 217860 [details] anki-2.1.33-preview.patch Here's a patch for preview and testing which updates Anki to 2.1.33. It also contains a slightly changed version of devel/py-maturin, which is also required to build the port. Current open items: - Although 2.1.33 is the latest version at the moment at this writing, an info will appear on startup that there's a new version available. If you all have some spare cycles to build and test Anki 2.1.33 that would be great.
Created attachment 217891 [details] anki-2.1.33-73-preview.patch Attached is a patch for Anki 2.1.33-73 which uses a more recent commit (of the upcoming 2.1.34 release) which fixes some issues with synchronizing via AnkiWeb and migrating existing configuration files from older installations: > Traceback (most recent call last): > File "/usr/local/share/anki/aqt/sync.py", line 93, in on_timer > on_normal_sync_timer(mw) > File "/usr/local/share/anki/aqt/sync.py", line 74, in on_normal_sync_timer > progress = mw.col.latest_progress() > File "/usr/local/share/anki/anki/collection.py", line 97, in > latest_progress > return Progress.from_proto(self.backend.latest_progress()) > File "/usr/local/share/anki/anki/rsbackend_gen.py", line 28, in > latest_progress > output.ParseFromString(self._run_command(1, input)) > File "/usr/local/share/anki/anki/rsbackend.py", line 262, in _run_command > err_bytes = bytes(e.args[0]) > IndexError: tuple index out of range Thanks to David for the detailed error reports and tests!
Version 2.1.35 came out recently. To test, I manually generated anki-2.1.35-node_modules.tgz, updated the distinfo, and bumped the version in the Makefile: # Patch to Makefile 5,6c5 < DISTVERSION= 2.1.33-73 < DISTVERSIONSUFFIX=-g340c749d --- > DISTVERSION= 2.1.35 10,11c9 < #DISTFILES= anki-${DISTVERSION}-node_modules.tgz:npmcache < DISTFILES= anki-2.1.33-node_modules.tgz:npmcache --- > DISTFILES= anki-${DISTVERSION}-node_modules.tgz:npmcache 351c349 < _MY_BUILDHASH= 340c749d --- > _MY_BUILDHASH= 84dcaa8 I ran into two problems. When compiling a clean build, I get a failure at: Command failed: npm --silent install chalk@^4.0.0 jsdoc@^3.6.3 tmp@^0.2.0 uglify-js@^3.7.7 espree@^7.0.0 escodegen@^1.13.0 estraverse@^5.1.0 gmake[3]: *** [Makefile:27: .build/proto] Error 1 gmake[3]: Leaving directory '/usr/obj/usr/ports/games/anki/work/anki-2.1.35/ts' gmake[2]: *** [Makefile:119: build] Error 2 gmake[2]: Leaving directory '/usr/obj/usr/ports/games/anki/work/anki-2.1.35' ===> Compilation failed unexpectedly. However, I can manually cd into /usr/obj/usr/ports/games/anki/work/anki-2.1.35/ts and run the npm command (minus the '--silent' option). The npm install succeeds (no unusual output, '0' exit code). If I cd back to /usr/ports/games/anki, I can then continue compilation, which now succeeds. However, when installing I get an error about missing files: pkg-static: Unable to access file /usr/obj/usr/ports/games/anki/work/stage/usr/local/share/anki/aqt_data/web/congrats.html:No such file or directory pkg-static: Unable to access file /usr/obj/usr/ports/games/anki/work/stage/usr/local/share/anki/aqt_data/web/congrats.js:No such file or directory *** Error code 74 I'm not sure where the actual problem is, but I removed both 'congrats.html' and 'congrats.js' from '/usr/obj/usr/ports/games/anki/work/.PLIST.mktmp' and from 'pkg-plist', and was then able to install. Anki seems to work fine, syncing, card creation, review, etc.
Created attachment 219101 [details] anki-2.1.35.patch (In reply to David Schlachter from comment #7) Thank you for the info which was very helpful to track down those issues and the additional testing. Attached is an updated patch for the 2.1.35 release of Anki and it also contains an improved "make-npm-cache" target that really generates a npm cache (= ${WRKSRC}/.npm) and not a pre-generated "node_modules" directory. The issue with the "npx pbjs" command was caused by additional dependencies that are required by the CLI component of the "protobufjs" package. Those packages couldn't be installed during the build phase due the environment variable "npm_config_offline=true" which prevents "npm" to open active network connections. The template file "congrats.html" (from which "congrats.js" is generated) isn't included in the 2.1.35 release for some reason as far I can tell. It seems that the buildhash needs to be exactly 8 digits otherwise the check for new versions always reports that a newer version is available.
Created attachment 219102 [details] py-maturin.patch Slightly improved patch for devel/py-maturin (fixes in pkg-descr and the handling of concurrent installations).
A commit references this bug: Author: kai Date: Tue Nov 10 12:56:43 UTC 2020 New revision: 554832 URL: https://svnweb.freebsd.org/changeset/ports/554832 Log: [NEW PORT] devel/py-maturin Build and publish crates with pyo3, rust-cpython and cffi bindings as well as Rust binaries as Python packages. This project is meant as a zero configuration replacement for setuptools-rust and milksnake. It supports building wheels for Python 3.5+ on FreeBSD and various other platforms, can upload them to PyPI and has basic PyPy support. WWW: https://github.com/PyO3/maturin PR: 247666 Reviewed by: tcberner Differential Revision: https://reviews.freebsd.org/D26960 Changes: head/devel/Makefile head/devel/py-maturin/ head/devel/py-maturin/Makefile head/devel/py-maturin/distinfo head/devel/py-maturin/files/ head/devel/py-maturin/files/patch-setup.py head/devel/py-maturin/pkg-descr
A commit references this bug: Author: kai Date: Tue Nov 10 13:01:05 UTC 2020 New revision: 554834 URL: https://svnweb.freebsd.org/changeset/ports/554834 Log: games/anki: Update to 2.1.35 Since the 2.1.17 release the build system has been significantly revised and contains many changes (e.g. a Rust library with Python bindings) that had to be adapted to the Ports framework. Kudos to David Schlachter for the additional testing and feedback! Changelog since 2.1.16 (too many changes to list them separately): https://changes.ankiweb.net/ PR: 247666 Reported by: David Schlachter Reviewed by: tcberner (earlier revision) Differential Revision: https://reviews.freebsd.org/D26961 Changes: head/games/anki/Makefile head/games/anki/distinfo head/games/anki/files/patch-Makefile head/games/anki/files/patch-anki_lang.py head/games/anki/files/patch-anki_mpv.py head/games/anki/files/patch-anki_sound.py head/games/anki/files/patch-aqt_qt.py head/games/anki/files/patch-pylib_Makefile head/games/anki/files/patch-qt_Makefile head/games/anki/files/patch-qt_runanki head/games/anki/files/patch-rslib_Cargo.toml head/games/anki/files/patch-rspy_Makefile head/games/anki/pkg-plist
(In reply to David Schlachter from comment #7) Committed, once again thank for your feedback and testing. I suspect that the next release of Anki could take a while again, because the build requires now "bazel" and the "orjson" Python package is now a mandatory requirement. Let's see what else comes before the official 2.1.36 release of Anki. :-)
orjson: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=266213