Bug 225659 - emacs flavor handling is wrong
Summary: emacs flavor handling is wrong
Status: Closed FIXED
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Some People
Assignee: Joseph Mingrone
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2018-02-04 10:28 UTC by Yasuhiro Kimura
Modified: 2018-03-13 00:12 UTC (History)
3 users (show)

See Also:
bugzilla: maintainer-feedback? (bdrewery)


Attachments
Build log of devel/magit. (8.69 KB, text/plain)
2018-02-04 10:28 UTC, Yasuhiro Kimura
no flags Details
Build log of www/emacs-w3m (10.86 KB, text/plain)
2018-02-04 14:47 UTC, Yasuhiro Kimura
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Yasuhiro Kimura freebsd_committer freebsd_triage 2018-02-04 10:28:34 UTC
Created attachment 190311 [details]
Build log of devel/magit.

I preferred editors/emacs-nox11 as emacs. So I added 'EMACS_PORT_NAME=emacs-nox11' to make.conf and built and installed my favorite emacs ports. It worked fine both on real environment and with poudriere. 

But after ports r460621 emacs ports are flavorized. So I replace 'EMACS_PORT_NAME=emacs-nox11' with 'DEFAULT_VERSIONS+=emacs=nox' in make.conf and rebuild and installed emacs ports. On real environment everything worked fine. But with poudriere build of some emacs ports failed because of dependency error. Attached file is build log of devel/magit. At first I faced this problem with ports-mgmt/poudriere. So I switched to ports-mgmt/poudriere-devel to check if it is fixed with development version. But it still happened.
Comment 1 Yasuhiro Kimura freebsd_committer freebsd_triage 2018-02-04 14:47:42 UTC
Created attachment 190315 [details]
Build log of www/emacs-w3m

Attached file is build log of www/emacs-w3m. Another example of dependency error.
Comment 2 elofu17 2018-02-05 20:27:15 UTC
(This is just a comment in case you (or someone else) has the same problem as I had)



I had sort of the same problem with emacs-nox11 and poudriere.

Solution: Just update poudriere to >v3.2.


Story:
Poudriere (<v3.2) complained that "editors/emacs-nox11" could not be found.
In UPDATING I found info telling me to set 'DEFAULT_VERSIONS+=emacs=nox', but that didn't work. Specifying port "editors/emacs" would depend on gtk and build tons of unwanted X11 packets.

Found some forum posts, recommending to upgrade poudriere.

So I upgraded it.
I removed the 'DEFAULT_VERSIONS+=emacs=nox' from make.conf.
I reverted the port name back to the old "editors/emacs-nox11".

Now poudirere works. It finds the port and compiles the nox version. :-)



I don't know why things still work or why the instructions in UPDATING were a bit misleading.
Comment 3 Bryan Drewery freebsd_committer freebsd_triage 2018-03-06 19:04:03 UTC
(In reply to elofu17 from comment #2)
> (This is just a comment in case you (or someone else) has the same problem
> as I had)
> 
> 
> 
> I had sort of the same problem with emacs-nox11 and poudriere.
> 
> Solution: Just update poudriere to >v3.2.
> 
> 
> Story:
> Poudriere (<v3.2) complained that "editors/emacs-nox11" could not be found.

Because it's not a valid port

# grep editors/emacs-nox11 MOVED
editors/emacs-nox11|editors/emacs@nox|2018-02-01|Moved to a flavored version


> In UPDATING I found info telling me to set 'DEFAULT_VERSIONS+=emacs=nox',
> but that didn't work. Specifying port "editors/emacs" would depend on gtk
> and build tons of unwanted X11 packets.
> 
> Found some forum posts, recommending to upgrade poudriere.
> 
> So I upgraded it.

Then editors/emacs@nox should work.

> I removed the 'DEFAULT_VERSIONS+=emacs=nox' from make.conf.
> I reverted the port name back to the old "editors/emacs-nox11".
> 
> Now poudirere works. It finds the port and compiles the nox version. :-)
> 
> 
> 
> I don't know why things still work or why the instructions in UPDATING were
> a bit misleading.


I haven't looked at this yet but my initial guess is that it's not a Poudriere
bug, and seeing more details I think even more so.

There's funky things going on with the emacs port. There is not even a proper
EMACS_DEFAULT defined in bsd.default-versions.mk:
    # Possible_values: full canna nox devel_full devel_nox
    #EMACS_DEFAULT?=        let the flavor be the default if not explicitly set

In the port itself I see these invalid patterns:
.if ${FLAVOR:U} == canna
...
.elif ${FLAVOR:U} == nox
...
.include <bsd.port.options.mk>

FLAVOR cannot be checked before including bsd.port.options.mk or bsd.port.pre.mk.

~/svn/ports/editors/emacs # make -V FLAVORS -V FLAVOR
full canna nox
full

Fine.

~/svn/ports/editors/emacs # EMACS_DEFAULT=nox make -V FLAVORS -V FLAVOR
full canna nox
full

Hmm.. Adding to make.conf:
~/svn/ports/editors/emacs # grep emacs /etc/make.conf
DEFAULT_VERSIONS+=emacs=nox
~/svn/ports/editors/emacs # make -V FLAVORS -V FLAVOR
full canna nox
full


Bad.

So the port is the problem, not Poudriere.

~/svn/ports/editors/emacs # make -V OPTIONS_EXCLUDE
[blank]

This is clearly wrong in the port.
Comment 4 Bryan Drewery freebsd_committer freebsd_triage 2018-03-06 19:07:08 UTC
https://wiki.freebsd.org/Ports/FlavorsMigration
Comment 5 Joseph Mingrone freebsd_committer freebsd_triage 2018-03-06 19:12:31 UTC
(In reply to Yasuhiro KIMURA from comment #0)
From the 20180201 UPDATING entry:

An alternative to the default flavor, full, for editors/emacs and editors/emacs-devel can be set in make.conf.  For example, to specify nox as the default flavor for both ports, use

     .if ${.CURDIR:M*/editors/emacs*}
     FLAVOR?=nox
     .endif

For ports using Emacs, an alternative to the default flavor, full, can be also be set in make.conf.  For example, to automatically depend on emacs-nox, use

     DEFAULT_VERSIONS+=emacs=nox
Comment 6 Mathieu Arnold freebsd_committer freebsd_triage 2018-03-06 19:13:04 UTC
I think the problem is was a FLAVORS vs OPTION conflicts.  This should be fixed as of ports r462411 (or close by) when the X11 option for the port was removed, and the X11 or not X11 became a FLAVORS only thing.
Comment 7 Bryan Drewery freebsd_committer freebsd_triage 2018-03-06 19:15:16 UTC
(In reply to Joseph Mingrone from comment #5)
> (In reply to Yasuhiro KIMURA from comment #0)
> From the 20180201 UPDATING entry:
> 
> An alternative to the default flavor, full, for editors/emacs and
> editors/emacs-devel can be set in make.conf.  For example, to specify nox as
> the default flavor for both ports, use
> 
>      .if ${.CURDIR:M*/editors/emacs*}
>      FLAVOR?=nox
>      .endif

This could work.

> 
> For ports using Emacs, an alternative to the default flavor, full, can be
> also be set in make.conf.  For example, to automatically depend on
> emacs-nox, use
> 
>      DEFAULT_VERSIONS+=emacs=nox

This can't work due to ${FLAVOR:U} used before bsd.port.*.mk; there is no FLAVOR defined until bsd.port.*.mk is included.  Poudriere relies on having 'make -V FLAVOR' and 'make -V OPTIONS_EXCLUDE' (etc) be correct for the default.  Because
FLAVOR isn't defined (and FLAVOR:U is admitting that) for the default from DFEFAULT_VERSIONS, it builds the wrong thing.
Comment 8 Bryan Drewery freebsd_committer freebsd_triage 2018-03-06 19:19:52 UTC
(In reply to Joseph Mingrone from comment #5)
> (In reply to Yasuhiro KIMURA from comment #0)
> From the 20180201 UPDATING entry:
> 
> An alternative to the default flavor, full, for editors/emacs and
> editors/emacs-devel can be set in make.conf.  For example, to specify nox as
> the default flavor for both ports, use
> 
>      .if ${.CURDIR:M*/editors/emacs*}
>      FLAVOR?=nox
>      .endif

Or using editors/emacs@nox passed to Poudriere.

> 
> For ports using Emacs, an alternative to the default flavor, full, can be
> also be set in make.conf.  For example, to automatically depend on
> emacs-nox, use
> 
>      DEFAULT_VERSIONS+=emacs=nox

Aha. Then perhaps it's fine. I think the real problem here is confusion about
what DEFAULT_VERSIONS applies to.
Comment 9 Joseph Mingrone freebsd_committer freebsd_triage 2018-03-06 19:21:54 UTC
(In reply to Bryan Drewery from comment #8)
This is all explained in the UPDATING entry.  Please let me know if something in that entry in not clear.
Comment 10 Bryan Drewery freebsd_committer freebsd_triage 2018-03-06 19:23:33 UTC
(In reply to Joseph Mingrone from comment #9)
> (In reply to Bryan Drewery from comment #8)
> This is all explained in the UPDATING entry.  Please let me know if
> something in that entry in not clear.

Well 2 users have been confused by it at least in this report. I think usually
DEFAULT_VERSIONS applies to the main port if possible as well. I think it could
here with some refactoring or helpers.
Comment 11 Bryan Drewery freebsd_committer freebsd_triage 2018-03-06 19:37:48 UTC
(In reply to Bryan Drewery from comment #10)
> (In reply to Joseph Mingrone from comment #9)
> > (In reply to Bryan Drewery from comment #8)
> > This is all explained in the UPDATING entry.  Please let me know if
> > something in that entry in not clear.
> 
> Well 2 users have been confused by it at least in this report. I think
> usually
> DEFAULT_VERSIONS applies to the main port if possible as well. I think it
> could
> here with some refactoring or helpers.

I'm probably wrong about DEFAULT_VERSIONS here again.
Comment 12 Joseph Mingrone freebsd_committer freebsd_triage 2018-03-11 18:43:13 UTC
Are there remaining problems? Can this be closed?
Comment 13 Yasuhiro Kimura freebsd_committer freebsd_triage 2018-03-12 05:53:42 UTC
(In reply to Joseph Mingrone from comment #12)

No. The problem is not solved at all. I updated port tree to ports r464215 and confirmed both 'poudriere testport devel/magit' and 'poudriere testport www/emacs-w3m' still fail with same error as before. Of course there is 'DEFAULT_VERSIONS+=emacs=nox' in ${PREFIX}/etc/poudriere.d/make.conf.

After all which is the source of this problem, poudriere or flavorized emacs ports?
Comment 14 Joseph Mingrone freebsd_committer freebsd_triage 2018-03-12 15:06:39 UTC
I had tested in the past, but was not able to reproduce the problem.  Today I tried again.  I am running poudriere-devel-3.2.99.20180309 on 11.1 release.

1. Put 'DEFAULT_VERSIONS+=emacs=nox' in ${PREFIX}/etc/poudriere.d/make.conf.
2. poudriere testport -j 11amd64 devel/magit: No error
3. poudriere testport -j 11i386 devel/magit: Problem reproduced
4. Start the jail and `make -C /usr/ports/devel/magit install`: No error
5. Stop the jail
6. Remove 'DEFAULT_VERSIONS+=emacs=nox' in ${PREFIX}/etc/poudriere.d/make.conf.
7. poudriere testport -j 11i386 devel/magit@nox: editors/with-editor@nox and devel/dash.el@nox are built and no errors building devel/magit@nox

So, there does seem to be a problem with 'DEFAULT_VERSIONS+=emacs=nox' and poudriere.  I will try and dig deeper.  In the meantime, it looks like if you specify the flavor on the command line it will avoid the problems.
Comment 15 Joseph Mingrone freebsd_committer freebsd_triage 2018-03-12 16:05:13 UTC
I am also able to reproduce the OPTIONS issue you described at https://github.com/freebsd/poudriere/issues/600.

With 'DEFAULT_VERSIONS+=emacs=nox' in /etc/make.conf

cd /usr/ports/devel/magit
make rmconfig-recursive
make -VFLAVOR
nox
make config-recursive [1]
make rmconfig-recursive
make FLAVOR=nox config-recursive [2]

[1] brings up options dialog for magit-emacs25_nox, and options dialog for emasc-25 and dependencies (incorrect)

[2] brings up options dialog for magit-emacs25_nox, and options dialog for emasc-nox-25 and dependencies (correct)
Comment 16 Bryan Drewery freebsd_committer freebsd_triage 2018-03-12 17:29:05 UTC
(In reply to Joseph Mingrone from comment #14)
> I had tested in the past, but was not able to reproduce the problem.  Today
> I tried again.  I am running poudriere-devel-3.2.99.20180309 on 11.1 release.
> 
> 1. Put 'DEFAULT_VERSIONS+=emacs=nox' in ${PREFIX}/etc/poudriere.d/make.conf.
> 2. poudriere testport -j 11amd64 devel/magit: No error
> 3. poudriere testport -j 11i386 devel/magit: Problem reproduced
> 4. Start the jail and `make -C /usr/ports/devel/magit install`: No error
> 5. Stop the jail
> 6. Remove 'DEFAULT_VERSIONS+=emacs=nox' in
> ${PREFIX}/etc/poudriere.d/make.conf.
> 7. poudriere testport -j 11i386 devel/magit@nox: editors/with-editor@nox and
> devel/dash.el@nox are built and no errors building devel/magit@nox
> 
> So, there does seem to be a problem with 'DEFAULT_VERSIONS+=emacs=nox' and
> poudriere.  I will try and dig deeper.  In the meantime, it looks like if
> you specify the flavor on the command line it will avoid the problems.

Yup there's definitely a problem here. Poudriere seems to get confused regarding
the default FLAVOR loses track of devel/dash.el@nox. Working on a fix.
Comment 17 Bryan Drewery freebsd_committer freebsd_triage 2018-03-12 17:34:35 UTC
The problem is that the FLAVORS is ordered differently than the Python reference
implementation. For Python the default FLAVOR always comes first but that's not
the case here. I'm not sure that we made that a rule but I'll try to fix it in
Poudriere anyhow.
Comment 18 commit-hook freebsd_committer freebsd_triage 2018-03-12 18:19:45 UTC
A commit references this bug:

Author: bdrewery
Date: Mon Mar 12 18:19:29 UTC 2018
New revision: 464267
URL: https://svnweb.freebsd.org/changeset/ports/464267

Log:
  FLAVORS: Fix several 'make *recursive' targets not passing along FLAVOR properly.

  PR:		225659 comment #15
  Approved by:	portmgr (implicit)

Changes:
  head/Mk/bsd.port.mk
Comment 19 Bryan Drewery freebsd_committer freebsd_triage 2018-03-12 18:26:26 UTC
(In reply to Bryan Drewery from comment #17)
> The problem is that the FLAVORS is ordered differently than the Python
> reference
> implementation. For Python the default FLAVOR always comes first but that's
> not
> the case here. I'm not sure that we made that a rule but I'll try to fix it
> in
> Poudriere anyhow.

I'm likely going to commit this patch to reorder FLAVORS because of [1] and [2] documenting the default being first.

Index: Mk/bsd.port.mk
===================================================================
--- Mk/bsd.port.mk      (revision 464267)
+++ Mk/bsd.port.mk      (working copy)
@@ -1492,6 +1492,11 @@ IGNORE=  Unknown flavor '${FLAVOR}', possible flavo
 FLAVOR=        ${FLAVORS:[1]}
 .endif

+# Reorder FLAVORS so the default is first if set by the port.
+.if empty(_FLAVOR) && !empty(FLAVORS) && !empty(FLAVOR)
+FLAVORS:=      ${FLAVOR} ${FLAVORS:N${FLAVOR}}
+.endif
+
 .if !empty(FLAVOR) && !defined(_DID_FLAVORS_HELPERS)
 _DID_FLAVORS_HELPERS=  yes
 _FLAVOR_HELPERS_OVERRIDE=      DESCR PLIST PKGNAMEPREFIX PKGNAMESUFFIX



[1] https://wiki.freebsd.org/Ports/FlavorsMigration
[2] https://wiki.freebsd.org/Ports/FlavorsTools
Comment 20 commit-hook freebsd_committer freebsd_triage 2018-03-12 18:31:00 UTC
A commit references this bug:

Author: bdrewery
Date: Mon Mar 12 18:29:58 UTC 2018
New revision: 464269
URL: https://svnweb.freebsd.org/changeset/ports/464269

Log:
  MFH: r464267

  FLAVORS: Fix several 'make *recursive' targets not passing along FLAVOR properly.

  PR:		225659 comment #15
  Approved by:	portmgr (implicit)

Changes:
_U  branches/2018Q1/
  branches/2018Q1/Mk/bsd.port.mk
Comment 21 commit-hook freebsd_committer freebsd_triage 2018-03-12 19:05:32 UTC
A commit references this bug:

Author: bdrewery
Date: Mon Mar 12 19:04:59 UTC 2018
New revision: 464273
URL: https://svnweb.freebsd.org/changeset/ports/464273

Log:
  FLAVORS: Reorder so the default is first.

  It is documented that the first FLAVOR is the default.  Due to some use
  of DEFAULT_VERSIONS and USES, the default FLAVOR may be determined
  dynamically.  Poudriere in particular expects the default to be first
  and does the wrong thing for emacs ports currently [1].  This resolves
  that and avoids other tools making the same mistake.

  PR:		225659 comment #14
  Approved by:	portmgr (implicit)

Changes:
  head/Mk/bsd.port.mk
Comment 22 commit-hook freebsd_committer freebsd_triage 2018-03-12 19:05:36 UTC
A commit references this bug:

Author: bdrewery
Date: Mon Mar 12 19:05:29 UTC 2018
New revision: 464274
URL: https://svnweb.freebsd.org/changeset/ports/464274

Log:
  MFH: r464273

  FLAVORS: Reorder so the default is first.

  It is documented that the first FLAVOR is the default.  Due to some use
  of DEFAULT_VERSIONS and USES, the default FLAVOR may be determined
  dynamically.  Poudriere in particular expects the default to be first
  and does the wrong thing for emacs ports currently [1].  This resolves
  that and avoids other tools making the same mistake.

  PR:		225659 comment #14
  Approved by:	portmgr (implicit)

Changes:
_U  branches/2018Q1/
  branches/2018Q1/Mk/bsd.port.mk
Comment 23 Yasuhiro Kimura freebsd_committer freebsd_triage 2018-03-13 00:12:30 UTC
I updated ports tree to ports r464327 and confirmed both 'poudriere testport deve/magit' and 'poudriere options devel/magit' work fine now.
Thank you for fixing them.