Bug 251191 - Mk/Uses/ninja.mk: 'USES=cmake emacs' results in build error with ninja
Summary: Mk/Uses/ninja.mk: 'USES=cmake emacs' results in build error with ninja
Status: Open
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Only Me
Assignee: Port Management Team
URL: https://reviews.freebsd.org/D27494
Depends on:
Reported: 2020-11-16 17:50 UTC by Yasuhiro Kimura
Modified: 2021-01-13 22:34 UTC (History)
3 users (show)

See Also:
yasu: exp-run?

Patch file (915 bytes, patch)
2020-11-16 17:50 UTC, Yasuhiro Kimura
no flags Details | Diff
Updated patch file (1.35 KB, patch)
2020-11-17 08:30 UTC, Yasuhiro Kimura
no flags Details | Diff
Updated patch file (64.97 KB, patch)
2020-12-06 22:32 UTC, Yasuhiro Kimura
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Yasuhiro Kimura freebsd_committer 2020-11-16 17:50:17 UTC
Created attachment 219737 [details]
Patch file

1. Summary

'USES=cmake emacs' results in build error with ninja. This is bug of ports build system. Attached patch fixes it.

2. Background

I'm trying to create port of following software.

libegit2: Emacs bindings for libgit2

3. What is the problem?

To port libegit2 following 2 steps are necessary.

a. Build libegit2.so with cmake and install it to ${PREFI}/lib
b. Byte-compile libgit.el and install them to

So I need to add 'USES=cmake emacs' in Makefile. But it causes error of ninja such as following.

===>  License GPLv3+ accepted by the user
===>   libegit2-emacs27_nox- depends on file: /usr/local/sbin/pkg - found
===> Fetching all distfiles required by libegit2-emacs27_nox- for building
===>  Extracting for libegit2-emacs27_nox-
=> SHA256 Checksum OK for magit-libegit2-
===>  Patching for libegit2-emacs27_nox-
===>  Applying FreeBSD patches for libegit2-emacs27_nox- from /usr0/freebsd/ports/git/devel/libegit2/files
===>   libegit2-emacs27_nox- depends on file: /usr/local/bin/cmake - found
===>   libegit2-emacs27_nox- depends on executable: ninja - found
===>   libegit2-emacs27_nox- depends on file: /usr/local/bin/emacs-27.1 - found
===>   libegit2-emacs27_nox- depends on package: pkgconf>=1.3.0_1 - found
===>   libegit2-emacs27_nox- depends on shared library: libgit2.so - found (/usr/local/lib/libgit2.so)
===>  Configuring for libegit2-emacs27_nox-
===>  Performing out-of-source build
/bin/mkdir -p /usr0/freebsd/ports/work/usr0/freebsd/ports/git/devel/libegit2/work-nox/.build
-- The C compiler identification is Clang 10.0.1
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- The CXX compiler identification is Clang 10.0.1
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found PkgConfig: pkgconf (found version "1.7.3") 
-- Checking for module 'libgit2'
--   Found libgit2, version 1.0.1
-- Configuring done
-- Generating done
CMake Warning:
  Manually-specified variables were not used by the project:


-- Build files have been written to: /usr0/freebsd/ports/work/usr0/freebsd/ports/git/devel/libegit2/work-nox/.build
===>  Building for libegit2-emacs27_nox-
ninja: error: unknown target 'EMACS=/usr/local/bin/emacs-27.1'
===> Compilation failed unexpectedly.
Try to set MAKE_JOBS_UNSAFE=yes and rebuild before reporting the failure to
the maintainer.
*** Error code 1

make: stopped in /usr0/freebsd/ports/git/devel/libegit2

Command exit status: 1

I tried to build libegit2 according to the steps described in its document and it succeeded without any error. Moreover I selected some ports that use 'USES=cmake', intentionally change them to 'USES=cmake emacs' and tried to build them. Then build failed with same error for all of them. So this is bug of ports build system.

4. Why the problem happens?

There is following line in Mk/Uses/emacs.mk


So if both 'cmake' and 'emacs' is added to USES, then ninja is invoked with something like `ninja -v EMACS=/usr/local/bin/emacs-27.1`. But according to the output of `ninja --help`, ninja doesn't accept such argument as 'NAME=VALUE' that `make` accepts. So it results in the error that happens when I try to build my devel/libegit2 port.

I checked Mk/Uses/*.mk and found following ones also add extra arguments to MAKE_ARGS.

* Mk/Uses/gnustep.mk
* Mk/Uses/scons.mk
* Mk/Uses/waf.mk

Moreover each port that uses 'USES=cmake' may do same thing. Therefore this problem should be fixed by changing Mk/Uses/ninja.mk.

5. Solution

In Mk/bsd.port.mk MAKE_ARGS is used in following targets.

* do-build
* do-install
* do-test

So the solution is to re-define them in Mk/Uses/ninja.mk. As for do-install and do-test, DO_MAKE_BUILD and DO_MAKE_TEST are defined and used when each target is called respectively. So re-define them in ninja.mk so ninja is directly invoked without using MAKE_ARGS. As for do-install DO_MAKE_INSTALL isn't defined. So re-define the target with same way in ninja.mk.

As the result re-definition of following variables in ninja.mk gets unnecessary.


So remove them.

Additionally define NINJA_ARGS whose default value is empty and add it to argument of executing ninja. It is intended that some ports may want to add extra argument when executing ninja.

6. Test

Following branch includes both devel/libegit2 port and this fix.

(Note: This branch will be removed once devel/libegit2 is committed.)

It can be build without any error. I also confirmed some ports that uses 'USES=cmake' can be built successfully with attached patch. But it should be tested more with exp-run.
Comment 1 Li-Wen Hsu freebsd_committer 2020-11-16 17:56:24 UTC
Over to maintainer
Comment 2 Tobias C. Berner freebsd_committer 2020-11-16 18:47:29 UTC
Moin moin 

You could try cmake:noninja in the meantime.


This flag to cmake.mk was added specifically to disable usage of ninja.

mfg Tobias
Comment 3 Yasuhiro Kimura freebsd_committer 2020-11-16 20:45:42 UTC
The problem is found. If a port uses 'USES=cmake' and also defines do-install target, then `make describe` produces such warning message as below.

make: "/usr0/freebsd/ports/head/sysutils/osquery/Makefile" line 78: warning: duplicate script for target "do-install" ignored
make: "/usr0/freebsd/ports/head/Mk/Uses/ninja.mk" line 29: warning: using previous script for "do-install" defined here

I'll investigate and update patch to fix it. So please suspend exp-run until new patch is unloaded.
Comment 4 Yasuhiro Kimura freebsd_committer 2020-11-17 08:30:05 UTC
Created attachment 219766 [details]
Updated patch file

Fix warning messages with following changes.

* In Mk/bsd.port.mk define DO_MAKE_INSTALL and use it in do-install target.
* In Mk/Uses/ninja.mk re-define DO_MAKE_INSTALL so ninja is directly invoked.

So please do exp-run with attached patch instead of original one.
Comment 5 Antoine Brodin freebsd_committer 2020-11-24 20:19:48 UTC
(In reply to Yasuhiro KIMURA from comment #4)

Can you open a review on https://reviews.freebsd.org/ ?
Comment 6 Yasuhiro Kimura freebsd_committer 2020-12-06 22:32:57 UTC
Created attachment 220329 [details]
Updated patch file

Change custom targets of ports using ninja so they don't use MAKE_{ARGS,CMD,ENV}.
Comment 7 Yasuhiro Kimura freebsd_committer 2020-12-06 22:46:38 UTC
(In reply to Antoine Brodin from comment #5)

Sorry for late response. I created following review.