Bug 194393

Summary: [exp-run] Mk/Uses/autoreconf.mk, add intltoolize support
Product: Ports & Packages Reporter: Olivier Duchateau <olivierd>
Component: Ports FrameworkAssignee: Tijl Coosemans <tijl>
Status: Closed FIXED    
Severity: Affects Only Me CC: cjpm, kwm, portmgr, samy.mahmoudi, tijl
Priority: --- Flags: tijl: exp-run?
Version: Latest   
Hardware: Any   
OS: Any   
Attachments:
Description Flags
Add intltoolize support in Mk/Uses/autoreconf.mk
none
patch
none
patch2
none
compiz-reloaded-core port tree shell archive (draft) none

Description Olivier Duchateau freebsd_committer freebsd_triage 2014-10-15 21:52:21 UTC
Created attachment 148358 [details]
Add intltoolize support in Mk/Uses/autoreconf.mk

If port requires USE_GNOME= intltool, build failed when we defined USES= autoreconf, because po/Makefile.in.in is not created (and other m4 files).

Patch add support of intltoolize, only when USE_GNOME= intltool is defined.
Comment 1 Olivier Duchateau freebsd_committer freebsd_triage 2014-10-15 21:53:11 UTC
*** Bug 194392 has been marked as a duplicate of this bug. ***
Comment 2 Tijl Coosemans freebsd_committer freebsd_triage 2014-10-16 19:52:24 UTC
Hi,

A few comments:

- You cannot use post-patch because then port makefiles cannot have a post-patch target.  In this case I think it's best to remove the entire target.  It's a real bug that maintainers need to be encouraged to fix upstream, so it's not good for USES=autoreconf to cover this up for them.

- The mkdir line needs to be removed as well.  The directory can be set in configure.ac using AC_CONFIG_MACRO_DIR and it's not always m4.  It's unfortunate that intltoolize does not create this directory like libtoolize does.  Ports will have to create this in their pre/post-patch or pre-configure targets if necessary.

- The arguments to intltoolize should be '-c -f' to force replacement of any existing files.  --automake silences some useful tests such as checking if configure.ac actually contains IT_PROG_INTLTOOL.
Comment 3 Koop Mast freebsd_committer freebsd_triage 2014-10-20 20:36:21 UTC
I cooked up the following before tijl@ pointed me to this bug.

https://reviews.freebsd.org/D976

We don't need to have a BUILD_DEPEND on intltool since USE_GNOME already does this for us. Apart from that we also need to check USE_GNOME for intlhack together with intltool. intlhack implies intltool but it won't show up in USE_GNOME, since it is a bsd.gnome.mk internal.

And I agree with tijl that the args should be "-c -f"
Comment 4 Tijl Coosemans freebsd_committer freebsd_triage 2014-10-20 20:45:07 UTC
Have you tested without the intlhack test?  USES are processed after bsd.gnome.mk so it should work.  There are other gnome components that imply intltool like gtk20.
Comment 5 Koop Mast freebsd_committer freebsd_triage 2014-10-21 13:53:21 UTC
I did not, and indeed hidden intltool depends all over the place from bsd.gnome.mk. So I think we should remove both the intltool and intlhack test and just run the intltoolize command if USE_GNOME is present.

irc/hexchat # make -V USE_GNOME
gtk20 libxml2 gconf2 gtk-update-icon-cache

No intltool present, but it shows up in make all-depends-list.
Comment 6 Tijl Coosemans freebsd_committer freebsd_triage 2014-10-21 14:47:08 UTC
Hmm, maybe use _USE_GNOME then?
Comment 7 Tijl Coosemans freebsd_committer freebsd_triage 2015-02-14 18:34:03 UTC
Created attachment 152981 [details]
patch

patch using _USE_GNOME
Comment 8 Tijl Coosemans freebsd_committer freebsd_triage 2015-02-14 18:35:51 UTC
Assign to portmgr for exp-run
Comment 9 Antoine Brodin freebsd_committer freebsd_triage 2015-02-14 18:44:10 UTC
Does USE_MATE need the same thing?
Comment 10 Tijl Coosemans freebsd_committer freebsd_triage 2015-02-14 19:31:51 UTC
No port uses USE_MATE=intltool and USE_MATE=intlhack.  I think it would be better to remove them.
Comment 11 Antoine Brodin freebsd_committer freebsd_triage 2015-02-15 23:44:47 UTC
10 new failures:

+ {"origin"=>"audio/terminatorx", "pkgname"=>"terminatorx-3.84_8", "phase"=>"configure", "errortype"=>"???"}
+ {"origin"=>"cad/gerbv", "pkgname"=>"gerbv-2.6.1_1", "phase"=>"configure", "errortype"=>"???"}
+ {"origin"=>"emulators/tme", "pkgname"=>"tme-0.8_5", "phase"=>"configure", "errortype"=>"???"}
+ {"origin"=>"games/exult", "pkgname"=>"exult-1.5.0.20130315_3", "phase"=>"configure", "errortype"=>"???"}
+ {"origin"=>"games/gtkpool", "pkgname"=>"gtkpool-0.5.0_6", "phase"=>"configure", "errortype"=>"???"}
+ {"origin"=>"games/lianliankan", "pkgname"=>"lianliankan-2.3.b1_7", "phase"=>"configure", "errortype"=>"???"}
+ {"origin"=>"games/xboard-devel", "pkgname"=>"xboard-devel-4.7.3.20140909.8", "phase"=>"configure", "errortype"=>"???"}
+ {"origin"=>"graphics/libpano13", "pkgname"=>"libpano13-2.9.18_4", "phase"=>"configure", "errortype"=>"???"}
+ {"origin"=>"japanese/jd", "pkgname"=>"ja-jd-2.8.8_4", "phase"=>"configure", "errortype"=>"???"}
+ {"origin"=>"net/mtr", "pkgname"=>"mtr-0.86", "phase"=>"configure", "errortype"=>"???"}

Failure logs:

http://package22.nyi.freebsd.org/data/101amd64-default-194393_197502/2015-02-14_23h59m51s/logs/errors/terminatorx-3.84_8.log
http://package22.nyi.freebsd.org/data/101amd64-default-194393_197502/2015-02-14_23h59m51s/logs/errors/gerbv-2.6.1_1.log
http://package22.nyi.freebsd.org/data/101amd64-default-194393_197502/2015-02-14_23h59m51s/logs/errors/tme-0.8_5.log
http://package22.nyi.freebsd.org/data/101amd64-default-194393_197502/2015-02-14_23h59m51s/logs/errors/exult-1.5.0.20130315_3.log
http://package22.nyi.freebsd.org/data/101amd64-default-194393_197502/2015-02-14_23h59m51s/logs/errors/gtkpool-0.5.0_6.log
http://package22.nyi.freebsd.org/data/101amd64-default-194393_197502/2015-02-14_23h59m51s/logs/errors/lianliankan-2.3.b1_7.log
http://package22.nyi.freebsd.org/data/101amd64-default-194393_197502/2015-02-14_23h59m51s/logs/errors/xboard-devel-4.7.3.20140909.8.log
http://package22.nyi.freebsd.org/data/101amd64-default-194393_197502/2015-02-14_23h59m51s/logs/errors/libpano13-2.9.18_4.log
http://package22.nyi.freebsd.org/data/101amd64-default-194393_197502/2015-02-14_23h59m51s/logs/errors/ja-jd-2.8.8_4.log
http://package22.nyi.freebsd.org/data/101amd64-default-194393_197502/2015-02-14_23h59m51s/logs/errors/mtr-0.86.log
Comment 12 Tijl Coosemans freebsd_committer freebsd_triage 2015-02-16 09:21:24 UTC
Created attachment 153027 [details]
patch2

Only run intltoolize if either configure.ac or configure.in contains '^(AC|IT)_PROG_INTLTOOL' because it returns an error otherwise.
Comment 13 Antoine Brodin freebsd_committer freebsd_triage 2015-02-17 10:00:47 UTC
Exp-run results:

http://package22.nyi.freebsd.org/build.html?mastername=101amd64-default-194393_197502&build=2015-02-16_10h12m53s

0 new failure,  looks good
Comment 14 commit-hook freebsd_committer freebsd_triage 2015-02-17 10:31:45 UTC
A commit references this bug:

Author: tijl
Date: Tue Feb 17 10:30:56 UTC 2015
New revision: 379150
URL: https://svnweb.freebsd.org/changeset/ports/379150

Log:
  Run intltoolize before running autoreconf when a port uses intltool and
  configure.ac (or configure.in) contains '^(AC|IT)_PROG_INTLTOOL'.

  Intltoolize updates intltool related build scripts and m4 files that are
  used by autoreconf to generate configure.  This change guarantees that
  the build scripts, configure and LOCALBASE/bin/intltool-* are all in sync.

  PR:		194393
  Differential Revision:	https://reviews.freebsd.org/D976
  Submitted by:	olivierd, kwm (modified)
  Exp-run by:	antoine
  Approved by:	portmgr (antoine)

Changes:
  head/Mk/Uses/autoreconf.mk
Comment 15 Samy Mahmoudi 2018-07-08 01:59:35 UTC
According to the tutorials below, intltoolize should be run after autoreconf, not before as meant in the previous log message. Could you confirm or refute that assertion ?

• https://developer.gnome.org/gtkmm-tutorial/stable/sec-internationalization-intro.html.enhttp://ptomato.name/advanced-gtk-techniques/html/gettext-project.html

Index: Mk/Uses/autoreconf.mk
===================================================================
--- Mk/Uses/autoreconf.mk	(révision 474149)
+++ Mk/Uses/autoreconf.mk	(copie de travail)
@@ -76,6 +76,7 @@
 # Don't modify time stamps if the files already exist
 	@test -e ${AUTORECONF_WRKSRC}/${f} || ${TOUCH} ${AUTORECONF_WRKSRC}/${f}
 .endfor
+	@(cd ${AUTORECONF_WRKSRC} && ${AUTORECONF} -f -i)
 .if defined(_USE_GNOME) && ${_USE_GNOME:Mintltool}
 	@(cd ${AUTORECONF_WRKSRC} && \
 		if test -f configure.ac; then configure=configure.ac; \
@@ -83,7 +84,6 @@
 		if ${EGREP} -q '^(AC|IT)_PROG_INTLTOOL' $${configure}; \
 		then ${LOCALBASE}/bin/intltoolize -f -c; fi)
 .endif
-	@(cd ${AUTORECONF_WRKSRC} && ${AUTORECONF} -f -i)
 .elif ! ${autoreconf_ARGS:Mbuild}
 IGNORE= Incorrect 'USES+=autoreconf:${autoreconf_ARGS}' expecting 'USES+=autoreconf[:build]'
 .endif
Comment 16 Tijl Coosemans freebsd_committer freebsd_triage 2018-07-08 09:25:57 UTC
Here's the actual code (with autoreconf after intltoolize):
https://gitlab.gnome.org/GNOME/gnome-common/blob/master/macros2/gnome-autogen.sh

Do you have a problem with some port or something?
Comment 17 Samy Mahmoudi 2018-07-08 13:24:20 UTC
Created attachment 194953 [details]
compiz-reloaded-core port tree shell archive (draft)
Comment 18 Samy Mahmoudi 2018-07-08 13:26:09 UTC
Thank you for your reply.

I am actually porting Compiz Reloaded (https://gitlab.com/compiz). I have already written draft files to port the core part (see attachements).

• The upstream autogen.sh seems to properly set up the build environment if I use CONFIGURE_SCRIPT="autogen.sh" in the Makefile. However, autogen.sh is run silently six times to produce _LATE_CONFIGURE_ARGS. Then, autogen.sh is run once more with the expected verbosity (revealed by make -d A configure).

   # Relevant lines of autogen.sh:
   mkdir -p m4
   aclocal -I m4 --install || exit 1
   autoreconf --verbose --force --install || exit 1
   intltoolize --copy --force --automake || exit 1
   
   cd "$OLDPWD" || exit $?
   if [ -z "$NOCONFIGURE" ]; then
       "$srcdir/configure" "$@" || exit 1
   fi

• If I add autoreconf to the USES variable, the generated configure script stops at some point with the error:

   config.status: error: po/Makefile.in.in was not created by intltoolize.

By trial and error, I have found out that reordering intltoolize and autoreconf worked around the problem. As a consequence, I looked on the web for a confirmation on the call order which (mis)leads me to the mentioned documentations. Now you confirmed intltoolize should be run before autoreconf, it definitely looks like upstream complexity: I have come to the conclusion that the presence of some macros in configure.ac implies call to autopoint which overrides po/Makefile.in.in that intltoolize has imported.

Therefore, I have two alternatives:
   • autoreconf -f -i && intltoolize -f -c
   • autopoint -f && AUTOPOINT='intltoolize -c' autoreconf -f -i

• I am resorting to define a pre-configure target (instead of reusing code with USES= autoreconf). Besides, I may also operate upstream as --disable-nls does not seem to drop dependency on gettext/intltool.
Comment 19 Tijl Coosemans freebsd_committer freebsd_triage 2018-07-08 13:41:40 UTC
(In reply to Samy Mahmoudi from comment #18)
Intltool and gettext are mutually exclusive.  You can normally patch configure.ac to remove gettext related macros.  You could submit this upstream but it may be better for upstream to drop intltool because it's obsolete (see https://wiki.gnome.org/MigratingFromIntltoolToGettext).

If the project uses glib it may not be important that --disable-nls works because you will depend on gettext (libintl) via glib anyway.  You can still have an NLS option in your port and prefix translation files in pkg-plist with %%NLS%%.
Comment 20 Samy Mahmoudi 2018-07-08 15:15:36 UTC
Regarding the NLS option, I have already written
   NLS_CONFIGURE_ENABLE=	nls
   NLS_USES=			gettext-runtime gettext-tools
   NLS_USE=			GNOME=intltool
and used %%NLS%% prefixes in pkg-plist, which lets the users to install translation files conditionnally.

However, keeping build dependencies on intltool or gettext while setting --disable-nls looked like an upstream dysfunction to me. It means users need to have intltool or gettext at build time whereas they have disabled NLS. What I have not realized and you pointed out is that one may nevertheless depend on gettext, via glib.

I agree with you: migrating from intltool to gettext upstream seems to be the best starting point.

Thank you for your responsiveness and your help.
Comment 21 Samy Mahmoudi 2018-07-08 15:17:50 UTC
conditionnally ---> conditionally
Comment 22 Tijl Coosemans freebsd_committer freebsd_triage 2018-07-08 15:57:27 UTC
(In reply to Samy Mahmoudi from comment #20)
You can't make intltool optional because you need intltoolize to generate the configure script.
Comment 23 Samy Mahmoudi 2018-07-08 19:15:24 UTC
Thank you very much for pointing this out. When I wrote,
 
   "However, keeping build dependencies on intltool or gettext while
   setting --disable-nls looked like an upstream dysfunction to me.
   It means users need to have intltool or gettext at build time whereas
   they have disabled NLS."

I thought like I was setting up the build environment to distribute a port that the users could just "configure", which is clearly not what my port do: the users will indeed set up their build environment with Autotools.

That is basically another reason why gettext has been deprecated in favour gettext-runtime and gettext-tools, so I could easily write something like:

   USES= ... gettext-runtime:build gettext-tools
   USE_GNOME= ... intltool

   ...

   NLS_CONFIGURE_ENABLE=	nls
   NLS_USES=			gettext-runtime:run
Comment 24 Samy Mahmoudi 2018-07-08 19:17:12 UTC
do ---> does