Bug 209302 - [exp-run] iconv.mk looks for ${LOCALBASE}/include/iconv.h before dependencies are installed
Summary: [exp-run] iconv.mk looks for ${LOCALBASE}/include/iconv.h before dependencies...
Status: Closed FIXED
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Many People
Assignee: Tijl Coosemans
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2016-05-05 18:58 UTC by Ravi Pokala
Modified: 2016-05-13 05:31 UTC (History)
1 user (show)

See Also:


Attachments
print/cups config.log for bad (within release.sh) and good (direct from command-line) builds (41.37 KB, application/x-gzip)
2016-05-09 20:13 UTC, Ravi Pokala
no flags Details
iconv.mk.patch (603 bytes, patch)
2016-05-09 21:20 UTC, Tijl Coosemans
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Ravi Pokala 2016-05-05 18:58:06 UTC
While building a release (with release(7)), I get this error:

----------------------------------------------------------------
echo Linking ippfind...
Linking ippfind...
cc -L../cgi-bin -L../cups -L../filter -L../ppdc -L../scheduler -fstack-protector -Wl,-R/usr/local/lib  -fPIE -pie -fPIC     -o ippfind ippfind.o -lcups  -L/usr/local/lib -lavahi-common -lavahi-client   -lz -lpthread -lm -lcrypt -L/usr/local/lib  -lz
../cups/libcups.so: undefined reference to `libiconv'
../cups/libcups.so: undefined reference to `libiconv_close'
../cups/libcups.so: undefined reference to `libiconv_open'
cc: error: linker command failed with exit code 1 (use -v to see invocation)
Makefile:182: recipe for target 'ippfind' failed
gmake[5]: *** [ippfind] Error 1
gmake[5]: *** Waiting for unfinished jobs....
1 warning generated.
gmake[5]: Leaving directory '/usr/ports/print/cups/work/cups-2.1.3/test'
Makefile:31: recipe for target 'all' failed
gmake[4]: *** [all] Error 1
gmake[4]: Leaving directory '/usr/ports/print/cups/work/cups-2.1.3'
===> Compilation failed unexpectedly.
----------------------------------------------------------------

src tree: base/head@r299122
ports tree: ports/head@r414649
Comment 1 Tijl Coosemans freebsd_committer 2016-05-05 21:45:54 UTC
What is the output of the following command:

make -C /usr/ports/print/cups -V CFLAGS

Does /usr/local/include/iconv.h exist?
Comment 2 Ravi Pokala 2016-05-05 21:55:40 UTC
> What is the output of the following command:
> 
> make -C /usr/ports/print/cups -V CFLAGS

    -O2 -pipe  -fstack-protector -fno-strict-aliasing

That's the case for both the host, and the chroot in which the build is being done.

> Does /usr/local/include/iconv.h exist?

It exists on the host, but not within the chroot.
Comment 3 Tijl Coosemans freebsd_committer 2016-05-06 09:51:12 UTC
If /usr/local/include/iconv.h exists on the host then the output of 'make -C /usr/ports/print/cups -V CFLAGS' should contain '-DLIBICONV_PLUG'.  The is flag is added in /usr/ports/Mk/Uses/iconv.mk.  You'll have to figure out why that isn't working.  Did you make any modifications to the ports tree?  Is there anything special in /etc/make.conf?
Comment 4 Ravi Pokala 2016-05-06 14:29:32 UTC
> The is flag is added in /usr/ports/Mk/Uses/iconv.mk.

Both the host and the chroot versions of that file are identical, and have:

    CPPFLAGS+=	-DLIBICONV_PLUG
    CFLAGS+=	-DLIBICONV_PLUG
    CXXFLAGS+=	-DLIBICONV_PLUG
    OBJCFLAGS+=	-DLIBICONV_PLUG

> Is there anything special in /etc/make.conf?

Neither the host nor the chroot even *has* an /etc/make.conf

> Did you make any modifications to the ports tree?

No. This is happening as part of a release build, using release(7), which checks out clean versions of the src, ports, and docs trees from SVN immediately before building. All inputs should be pristine.

This is easily reproducible:

  - Sync src sandbox at ${SRCROOT}
  - Copy ${SRCROOT}/release/release.conf.sample to /tmp/release.conf
  - Edit /tmp/release.conf
    - Change "CHROOTDIR" to point to desired output location
  - cd ${SRCROOT}/release
  - sudo ./release.sh -c /tmp/release.conf
Comment 5 Tijl Coosemans freebsd_committer 2016-05-06 17:46:28 UTC
What is the output of "/usr/bin/uname -U" on the host and in the chroot?
Comment 6 Ravi Pokala 2016-05-06 18:24:18 UTC
The host reports 1100105; I'm not sure how to get that for the chroot, because the chroot is created in the context of release(7), and I don't know how to get a shell in that environment.

Looking at uname.c, `uname -U' returns the value of __FreeBSD_version for the binary; the src tree in the chroot has __FreeBSD_version set to 1100107.
Comment 7 Tijl Coosemans freebsd_committer 2016-05-07 10:55:30 UTC
Does /usr/include/iconv.h exist on the host and in the chroot?
What is the output of this:

readelf -s /lib/libc.so.7 | grep iconv

readelf -s $CHROOTDIR/lib/libc.so.7 | grep iconv
Comment 8 Ravi Pokala 2016-05-08 22:29:42 UTC
Host:

   113: 00000000000606d0    10 FUNC    GLOBAL DEFAULT   11 iconv_close@FBSD_1.3 (5)
   124: 00000000000609e0   592 FUNC    GLOBAL DEFAULT   11 __bsd_iconvlist@@FBSD_1.3 (5)
   191: 0000000000060910   142 FUNC    GLOBAL DEFAULT   11 __bsd___iconv@@FBSD_1.3 (5)
   355: 00000000000609d0    10 FUNC    GLOBAL DEFAULT   11 __bsd___iconv_free_list@@FBSD_1.3 (5)
   375: 0000000000060680    10 FUNC    GLOBAL DEFAULT   11 __iconv@FBSD_1.2 (4)
   615: 0000000000060690    10 FUNC    GLOBAL DEFAULT   11 __iconv_free_list@FBSD_1.2 (4)
   653: 0000000000060700    10 FUNC    GLOBAL DEFAULT   11 iconv_set_relocation_prefix@FBSD_1.3 (5)
   773: 0000000000060730     9 FUNC    GLOBAL DEFAULT   11 __bsd_iconv_open@@FBSD_1.3 (5)
  1213: 0000000000060720    10 FUNC    GLOBAL DEFAULT   11 iconvlist@FBSD_1.3 (5)
  1278: 0000000000060890   125 FUNC    GLOBAL DEFAULT   11 __bsd_iconv@@FBSD_1.3 (5)
  1370: 0000000000060c40    10 FUNC    GLOBAL DEFAULT   11 __bsd_iconv_canonicalize@@FBSD_1.3 (5)
  1714: 00000000000606c0    10 FUNC    GLOBAL DEFAULT   11 iconv_canonicalize@FBSD_1.2 (4)
  2138: 00000000000606b0    10 FUNC    GLOBAL DEFAULT   11 iconv@FBSD_1.3 (5)
  2303: 0000000000060710    10 FUNC    GLOBAL DEFAULT   11 iconvctl@FBSD_1.3 (5)
  2314: 00000000000606e0    10 FUNC    GLOBAL DEFAULT   11 iconv_open@FBSD_1.3 (5)
  2433: 0000000000395e6c     4 OBJECT  GLOBAL DEFAULT   25 _iconv_version@FBSD_1.3 (5)
  2457: 0000000000060c50   359 FUNC    GLOBAL DEFAULT   11 __bsd_iconvctl@@FBSD_1.3 (5)
  2537: 00000000000609a0    41 FUNC    GLOBAL DEFAULT   11 __bsd___iconv_get_list@@FBSD_1.3 (5)
  2583: 0000000000060840    25 FUNC    GLOBAL DEFAULT   11 __bsd_iconv_open_into@@FBSD_1.3 (5)
  2612: 0000000000060dc0     6 FUNC    GLOBAL DEFAULT   11 __bsd_iconv_set_relocation_prefix@@FBSD_1.3 (5)
  2807: 00000000000606f0    10 FUNC    GLOBAL DEFAULT   11 iconv_open_into@FBSD_1.3 (5)
  2965: 0000000000060860    41 FUNC    GLOBAL DEFAULT   11 __bsd_iconv_close@@FBSD_1.3 (5)
  3013: 00000000000606a0    13 FUNC    GLOBAL DEFAULT   11 __iconv_get_list@FBSD_1.2 (4)



chroot:

   113: 00000000000606a0    10 FUNC    GLOBAL DEFAULT   11 iconv_close@FBSD_1.3 (5)
   124: 00000000000609b0   592 FUNC    GLOBAL DEFAULT   11 __bsd_iconvlist@@FBSD_1.3 (5)
   191: 00000000000608e0   142 FUNC    GLOBAL DEFAULT   11 __bsd___iconv@@FBSD_1.3 (5)
   355: 00000000000609a0    10 FUNC    GLOBAL DEFAULT   11 __bsd___iconv_free_list@@FBSD_1.3 (5)
   375: 0000000000060650    10 FUNC    GLOBAL DEFAULT   11 __iconv@FBSD_1.2 (4)
   615: 0000000000060660    10 FUNC    GLOBAL DEFAULT   11 __iconv_free_list@FBSD_1.2 (4)
   653: 00000000000606d0    10 FUNC    GLOBAL DEFAULT   11 iconv_set_relocation_prefix@FBSD_1.3 (5)
   773: 0000000000060700     9 FUNC    GLOBAL DEFAULT   11 __bsd_iconv_open@@FBSD_1.3 (5)
  1213: 00000000000606f0    10 FUNC    GLOBAL DEFAULT   11 iconvlist@FBSD_1.3 (5)
  1278: 0000000000060860   125 FUNC    GLOBAL DEFAULT   11 __bsd_iconv@@FBSD_1.3 (5)
  1370: 0000000000060c10    10 FUNC    GLOBAL DEFAULT   11 __bsd_iconv_canonicalize@@FBSD_1.3 (5)
  1714: 0000000000060690    10 FUNC    GLOBAL DEFAULT   11 iconv_canonicalize@FBSD_1.2 (4)
  2138: 0000000000060680    10 FUNC    GLOBAL DEFAULT   11 iconv@FBSD_1.3 (5)
  2303: 00000000000606e0    10 FUNC    GLOBAL DEFAULT   11 iconvctl@FBSD_1.3 (5)
  2314: 00000000000606b0    10 FUNC    GLOBAL DEFAULT   11 iconv_open@FBSD_1.3 (5)
  2433: 000000000039559c     4 OBJECT  GLOBAL DEFAULT   25 _iconv_version@FBSD_1.3 (5)
  2457: 0000000000060c20   359 FUNC    GLOBAL DEFAULT   11 __bsd_iconvctl@@FBSD_1.3 (5)
  2537: 0000000000060970    41 FUNC    GLOBAL DEFAULT   11 __bsd___iconv_get_list@@FBSD_1.3 (5)
  2583: 0000000000060810    25 FUNC    GLOBAL DEFAULT   11 __bsd_iconv_open_into@@FBSD_1.3 (5)
  2612: 0000000000060d90     6 FUNC    GLOBAL DEFAULT   11 __bsd_iconv_set_relocation_prefix@@FBSD_1.3 (5)
  2807: 00000000000606c0    10 FUNC    GLOBAL DEFAULT   11 iconv_open_into@FBSD_1.3 (5)
  2965: 0000000000060830    41 FUNC    GLOBAL DEFAULT   11 __bsd_iconv_close@@FBSD_1.3 (5)
  3013: 0000000000060670    13 FUNC    GLOBAL DEFAULT   11 __iconv_get_list@FBSD_1.2 (4)
Comment 9 Tijl Coosemans freebsd_committer 2016-05-09 10:44:16 UTC
And does /usr/include/iconv.h exist?
Comment 10 Ravi Pokala 2016-05-09 14:04:53 UTC
> And does /usr/include/iconv.h exist?

Yes - identical versions are present both on the host and in the chroot.
Comment 11 Tijl Coosemans freebsd_committer 2016-05-09 14:18:28 UTC
What is the output of this command:

make -C /usr/ports/print/cups -V ICONV_PREFIX -V LOCALBASE -V ICONV_INCLUDE_PATH
Comment 12 Ravi Pokala 2016-05-09 14:28:27 UTC
For both host and chroot, it lists

    /usr
    /usr/local
    /usr/include
Comment 13 Tijl Coosemans freebsd_committer 2016-05-09 15:32:54 UTC
Does /usr/ports/Mk/Uses/iconv.mk contain these exact lines:

.if (${OPSYS} == FreeBSD && (${OSVERSION} < 1001514 \
 || (${OSVERSION} >= 1100000 && ${OSVERSION} < 1100069))) \
 || exists(${LOCALBASE}/include/iconv.h)
BUILD_DEPENDS+= libiconv>=1.14_9:converters/libiconv
CPPFLAGS+=      -DLIBICONV_PLUG
CFLAGS+=        -DLIBICONV_PLUG
CXXFLAGS+=      -DLIBICONV_PLUG
OBJCFLAGS+=     -DLIBICONV_PLUG
ICONV_INCLUDE_PATH=     ${LOCALBASE}/include
.endif

Because you said that /usr/local/include/iconv.h exists on your system the exists(...) test above should be true, but it isn't for some reason.  Can you double-check that /usr/local/include/iconv.h indeed exists.
Comment 14 Ravi Pokala 2016-05-09 15:42:09 UTC
> Does /usr/ports/Mk/Uses/iconv.mk contain these exact lines:

Yes, it does.

> Can you double-check that /usr/local/include/iconv.h indeed exists.

Huh. When you asked this earlier:

>> Does /usr/local/include/iconv.h exist?
> 
> It exists on the host, but not within the chroot.

But when I look now, it's the opposite - it exists within the chroot, not the host. I must have mis-read it initially. Sorry about that. :-(
Comment 15 Tijl Coosemans freebsd_committer 2016-05-09 16:20:17 UTC
Ok, assuming there's still a ports tree under CHROOTDIR can you run this command:

chroot CHROOTDIR make -C /usr/ports/print/cups -V ICONV_PREFIX -V LOCALBASE -V ICONV_INCLUDE_PATH -V CFLAGS
Comment 16 Ravi Pokala 2016-05-09 16:26:07 UTC
In the chroot:

    /usr
    /usr/local
    /usr/local/include
    -O2 -pipe  -DLIBICONV_PLUG -fstack-protector -fno-strict-aliasing


and on the host, for comparison:

    /usr
    /usr/local
    /usr/include
    -O2 -pipe  -fstack-protector -fno-strict-aliasing
Comment 17 Ravi Pokala 2016-05-09 18:57:32 UTC
BTW, if you think this is a matter of a screwed-up host, and not a real bug, then I'll just rebuild the host. I was actually running `release.sh' to generate the memstick to re-image the machine, but I can always fall back to using the most recent snapshot.
Comment 18 Tijl Coosemans freebsd_committer 2016-05-09 19:27:05 UTC
I don't know yet what the problem is.  So far everything looks normal.

Can you attach CHROOT/usr/ports/print/cups/work/cups-2.1.3/config.log to this bug (if the file exists)?

After that, try building cups in the chroot:

chroot CHROOTDIR make -C /usr/ports/print/cups clean
chroot CHROOTDIR make -C /usr/ports/print/cups install

If it fails please attach the new CHROOT/usr/ports/print/cups/work/cups-2.1.3/config.log as well.
Comment 19 Ravi Pokala 2016-05-09 20:13:04 UTC
Created attachment 170157 [details]
print/cups config.log for bad (within release.sh) and good (direct from command-line) builds

Executing the chroot by hand, the build succeeded. Comparing the two config.logs, the main things that look different are: (a) additional entries in $PATH and (b) additional -DLIBICONV_PLUG flag

`release.sh', in addition to running inside a chroot, also runs with a canonical environment; When I use `sudo' to run `make' within the $CHROOTDIR, it looks like it inherits my non-canonical $PATH.
Comment 20 Tijl Coosemans freebsd_committer 2016-05-09 21:20:30 UTC
Created attachment 170159 [details]
iconv.mk.patch

I think I know what's going on.  When the print/cups build is started via release.sh /usr/local/include/iconv.h does not exist yet.  It is installed by one of the dependencies of cups (Avahi, which is optional and has recently been enabled by default).  The problem is that the exists(...) test in iconv.mk runs before dependencies are installed.  There's no way to run this test after dependencies as far as I know, so this patch modifies iconv.mk such that -DLIBICONV_PLUG is added to CFLAGS unconditionally.  You can test it by modifying release.sh like this:

--- release/release.sh	(revision 299059)
+++ release/release.sh	(working copy)
@@ -226,6 +226,7 @@ chroot_setup() {
 	fi
 	if [ -z "${NOPORTS}" ] && [ -z "${PORTS_UPDATE_SKIP}" ]; then
 		${VCSCMD} ${PORTBRANCH} ${CHROOTDIR}/usr/ports
+		patch -d ${CHROOTDIR}/usr/ports < path/to/iconv.mk.patch
 	fi
 
 	if [ -z "${CHROOTBUILD_SKIP}" ]; then
Comment 21 Ravi Pokala 2016-05-10 08:30:14 UTC
SUCCESS!

===>  Configuring for cups-2.1.3_2
...
===>  Building for cups-2.1.3_2
...
===>  Staging for cups-2.1.3_2
...
===>   Generating temporary packing list
===> Creating groups.
===> Creating users
...
===>   Registering installation for cups-2.1.3_2 as automatic
Installing cups-2.1.3_2...
===> Creating groups.
Creating group 'cups' with gid '193'.
===> Creating users
Creating user 'cups' with uid '193'.
===> SECURITY REPORT: 
...
===>   ghostscript9-agpl-base-9.16_5 depends on shared library: libcupsimage.so - found (/usr/local/lib/libcupsimage.so)
Comment 22 Tijl Coosemans freebsd_committer 2016-05-10 13:10:36 UTC
Assign to portmgr for exp-run.

The exists(${LOCALBASE}/include/iconv.h) test in Mk/Uses/iconv.mk runs before dependencies are installed.  If one of the dependencies installs this file CPPFLAGS does not contain -DLIBICONV_PLUG and the port may fail to build.  The patch makes iconv.mk always add -DLIBICONV_PLUG (if the base system supports iconv).
Comment 23 Antoine Brodin freebsd_committer 2016-05-10 13:35:48 UTC
Can you confirm that the exp-run needs to be done on 10.2 or 10.3  (and that nothing changes for 9.3 or 10.1)
Comment 24 Tijl Coosemans freebsd_committer 2016-05-10 14:35:10 UTC
Ah, right, 9.3 never uses -DLIBICONV_PLUG and 10.1 always uses it.  Maybe no exp-run is needed then.  We've fixed problems with always using -DLIBICONV_PLUG on FreeBSD 10 for ports r384038.  If you do want to do an exp-run it should be at least 10.2.
Comment 25 Antoine Brodin freebsd_committer 2016-05-12 11:35:04 UTC
Exp-run results:

http://package23.nyi.freebsd.org/build.html?mastername=103i386-default-PR209302&build=2016-05-12_07h18m47s

0 new failure, looks good.
Comment 26 commit-hook freebsd_committer 2016-05-12 13:37:13 UTC
A commit references this bug:

Author: tijl
Date: Thu May 12 13:36:46 UTC 2016
New revision: 415064
URL: https://svnweb.freebsd.org/changeset/ports/415064

Log:
  Always add -DLIBICONV_PLUG to CPPFLAGS and friends when the base system
  iconv(3) is used.  Adding it only when LOCALBASE/include/iconv.h exists is
  not enough because this is tested before dependencies are installed and
  they may pull in converters/libiconv.

  PR:		209302
  Exp-run by:	antoine
  Approved by:	portmgr (antoine)

Changes:
  head/Mk/Uses/iconv.mk
Comment 27 Ravi Pokala 2016-05-13 05:31:04 UTC
Confirmed, a pristine version of `release.sh' runs to completion now. Thanks!