Bug 202838 - [exp-run] remove iconv*() from libiconv
Summary: [exp-run] remove iconv*() from libiconv
Status: Closed FIXED
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Ports Framework (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Only Me
Assignee: Tijl Coosemans
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2015-09-02 16:59 UTC by Tijl Coosemans
Modified: 2015-10-19 19:34 UTC (History)
2 users (show)

See Also:


Attachments
patch (17.25 KB, patch)
2015-09-02 16:59 UTC, Tijl Coosemans
no flags Details | Diff
patch2 (17.01 KB, patch)
2015-09-07 14:42 UTC, Tijl Coosemans
no flags Details | Diff
patch3 (25.94 KB, patch)
2015-09-09 16:39 UTC, Tijl Coosemans
no flags Details | Diff
patch4 (26.65 KB, patch)
2015-09-10 17:49 UTC, Tijl Coosemans
no flags Details | Diff
patch5 (25.71 KB, patch)
2015-10-05 13:15 UTC, Tijl Coosemans
no flags Details | Diff
patch6 (29.92 KB, patch)
2015-10-09 19:33 UTC, Tijl Coosemans
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Tijl Coosemans freebsd_committer freebsd_triage 2015-09-02 16:59:19 UTC
Created attachment 160640 [details]
patch

Spin-off from bug 202798

Remove iconv(), iconv_open() and iconv_close() symbols from libiconv.  Upstream never had these and now that they are in libc they make it more complicated to test for libc iconv support and prefer it over libiconv when both are available.

Additional fixes:
audio/deadbeef: remove LIBICONV_PLUG from a source file.  It's a compile-time option and should not be set in source code
comms/hidapi: use standard AM_ICONV to find iconv
deskutils/fbreader: let ports framework handle LIBICONV_PLUG
deskutils/ljclive: override test for iconv()
deskutils/owncloudclient: add USES=iconv and patch away test for iconv()
emulators/vmw: replace OSVERSION checks with ICONV_LIB checks and include <iconv.h> instead of <sys/iconv.h>
irc/scrollz: override test for iconv()
japanese/chasen-base: override test for iconv() and prevent linking with libiconv when libc supports iconv
net/c3270: override test for iconv support
www/htmlcxx: look for libiconv_open() instead of iconv_open() to find libiconv
www/httrack: look for libiconv() instead of iconv() to find libiconv
www/xapian-omega: override test for iconv()
x11/mrxvt(-devel): add USES=iconv and look for libiconv_open() instead of iconv_open() to find libiconv
x11/x3270: override test for iconv support

Not fixed:
japanese/uim-mozc: I cannot really figure this out, but it seems unrelated.  Beefy1 doesn't seem to build this port, so I couldn't find a baseline build log.
java/jboss71: out of memory error
www/p5-Gtk2-WebKit: seems to be a c++11 problem
Comment 1 Antoine Brodin freebsd_committer freebsd_triage 2015-09-07 13:09:40 UTC
Can you refresh the patch?  I have rejects on   x11/mrxvt/Makefile
Comment 2 Tijl Coosemans freebsd_committer freebsd_triage 2015-09-07 14:42:52 UTC
Created attachment 160802 [details]
patch2
Comment 4 Antoine Brodin freebsd_committer freebsd_triage 2015-09-09 06:38:59 UTC
Comparing output of pkg rquery "%o %B" before/after the patch on 9.3 i386,  I have this difference:

-devel/aegis libiconv.so.2
-devel/libexplain libiconv.so.2
-devel/sdl20 libiconv.so.2
-japanese/eb libcharset.so.1
-japanese/eb libiconv.so.2
-japanese/eblook libiconv.so.2
-java/jikes libiconv.so.2
-multimedia/transcode libiconv.so.2
-net-p2p/transmission-qt4 libiconv.so.2
-x11-wm/jwm libiconv.so.2

Build logs are at:
http://pb2.nyi.freebsd.org/jail.html?mastername=93i386-default-PR202838
Comment 5 Tijl Coosemans freebsd_committer freebsd_triage 2015-09-09 16:39:35 UTC
Created attachment 160867 [details]
patch3

comms/hidapi: use pre-configure instead of post-patch
emulators/vmw: add -I${LOCALBASE}/include to CFLAGS

-devel/aegis libiconv.so.2
This is ok, it only links libiconv because it may be needed to link libintl on
some systems.

-devel/libexplain libiconv.so.2
Fixed

-devel/sdl20 libiconv.so.2
Fixed

-japanese/eb libcharset.so.1
Replaced with nl_langinfo(3)

-japanese/eb libiconv.so.2
Fixed

-japanese/eblook libiconv.so.2
Fixed

-java/jikes libiconv.so.2
Fixed

-multimedia/transcode libiconv.so.2
This is ok.  The configure test for iconv_open added -liconv to LIBS so
everything linked with it, but only the subtitler plugin uses iconv and it
isn't enabled by default.

-net-p2p/transmission-qt4 libiconv.so.2
Fixed

-x11-wm/jwm libiconv.so.2
Fixed

I changed some of the previous fixes to make them easier to remove when FreeBSD 9 reaches EoL, so a full exp-run is needed.  If it's successful also one on FreeBSD 10 to be sure.
Comment 7 Tijl Coosemans freebsd_committer freebsd_triage 2015-09-10 17:49:15 UTC
Created attachment 160902 [details]
patch4

java/jikes: use the correct name of a configure variable
net-p2p/transmission-qt4: the change is in transmission-cli and wasn't included in the previous patch
Comment 8 Antoine Brodin freebsd_committer freebsd_triage 2015-09-10 23:05:12 UTC
On 9.3:  everything looked good.


On 10.1:

I launched exp-run on modified ports at http://pb2.nyi.freebsd.org/build.html?mastername=101i386-default-PR202838&build=2015-09-10_22h01m19s

So far,  there are changes with:
devel/sdl20 : now links against libiconv (causes a few failures)
devel/libexplain : now links against libiconv (causes dnsutl failure)
www/htmlcxx : now links against libiconv
multimedia/transcode : now links against libiconv
Comment 9 Antoine Brodin freebsd_committer freebsd_triage 2015-09-11 05:52:52 UTC
In the output of pkg rquery "%o %B" of the modified ports,  I have those differences between 10.1 i386 before patch and 10.1 i386 after patch:

+devel/libexplain libiconv.so.2
+devel/sdl20 libiconv.so.2
+multimedia/transcode libiconv.so.2
+www/htmlcxx libiconv.so.2
+www/httrack libiconv.so.2
+x11/mrxvt-devel libiconv.so.2
Comment 10 Tijl Coosemans freebsd_committer freebsd_triage 2015-10-05 13:15:18 UTC
Created attachment 161728 [details]
patch5

With iconv(), iconv_open() and iconv_close() removed from libiconv I thought that configure tests to find these functions in libiconv would fail, but they still succeed because the functions are in libc.  The previous patch removed a few things from port Makefiles based on that assumption.  This patch adds them back.  None of the changes affect FreeBSD 9 so an exp-run on 10 should be enough.
Comment 11 Antoine Brodin freebsd_committer freebsd_triage 2015-10-09 12:03:24 UTC
There is 1 new failure on 10.2: deskutils/fbreader

http://package18.nyi.freebsd.org/data/102amd64-default-PR202838/2015-10-07_21h03m47s/logs/errors/fbreader-0.99.6_2.log

On 10.*, those 3 ports used to link against libiconv.so.2 but no longer do with the patch:

net/samba4 libiconv.so.2
net/samba41 libiconv.so.2
net/samba42 libiconv.so.2

On 10.2, those 3 ports no longer link against libiconv.so.2  (they do link against it on 10.1):

net-mgmt/icinga-classicweb libiconv.so.2
net-mgmt/netxms libiconv.so.2
net/asterisk11 libiconv.so.2

If you want to compare 10.1 and 10.2 logs,  it's http://package22.nyi.freebsd.org/build.html?mastername=101i386-default-PR202838&build=2015-10-07_21h03m38s vs http://package18.nyi.freebsd.org/build.html?mastername=102amd64-default-PR202838&build=2015-10-07_21h03m47s
Comment 12 Tijl Coosemans freebsd_committer freebsd_triage 2015-10-09 19:33:23 UTC
Created attachment 161860 [details]
patch6

deskutils/fbreader: Move a command that uses CFLAGS from post-patch to pre-configure.  Build dependencies are installed after patching and they can pull in libiconv which affects CFLAGS in Mk/Uses/iconv.mk.

net/samba4*: Just bump PORTREVISION.  I believe these ports work as intended now.  The configure script will always add -liconv to linker flags when it happens to be installed which would be wrong but later on binaries are linked with -Wl,--as-needed and the linker discards -liconv because it finds iconv*() functions in libc now and no longer in libiconv.

The next three don't link with libiconv on FreeBSD 10.2 because base system iconv.h can be used again so libiconv isn't installed as a build dependency.

net-mgmt/icinga-classicweb: Remove dependency on iconv.  This port only links with libiconv because it may be needed to link libgd on some platforms.  I don't think that's the case on FreeBSD and if it is it needs to be fixed in graphics/gd.

net-mgmt/netxms: Patch configure script so it no longer adds -liconv to linker flags just because it happens to be installed.

net/asterisk11: Patch configure script so it no longer adds -liconv to linker flags just because it happens to be installed.
Comment 13 Antoine Brodin freebsd_committer freebsd_triage 2015-10-10 06:50:05 UTC
looks good
Comment 14 commit-hook freebsd_committer freebsd_triage 2015-10-10 14:03:33 UTC
A commit references this bug:

Author: tijl
Date: Sat Oct 10 14:03:05 UTC 2015
New revision: 398996
URL: https://svnweb.freebsd.org/changeset/ports/398996

Log:
  Remove iconv(), iconv_open() and iconv_close() symbols from libiconv.

  These were FreeBSD specific aliases for libiconv(), libiconv_open() and
  libiconv_close() that are now also provided by libc which complicates
  writing configure tests that work correctly when both libc iconv and
  libiconv are available.

  Also, because the libiconv iconv.h header redefines iconv* to libiconv*
  correct use of the header implies that the aliases aren't used.

  The following ports needed fixes because there was something wrong with
  the way they tried to detect or use iconv:

  audio/deadbeef: Remove LIBICONV_PLUG from a source file.  It's a
  compile-time option and should not be set in source code.
  comms/hidapi: Use standard AM_ICONV configure macro to look for iconv.
  deskutils/fbreader: Let ports framework deal with LIBICONV_PLUG.
  deskutils/ljclive: Override configure test for iconv.
  deskutils/owncloudclient: Add USES=iconv and patch test for iconv.
  devel/aegis: Bump PORTREVISION because it no longer uses libiconv.
  devel/libexplain: Add USES=iconv and override test for iconv.
  devel/sdl20: Override configure test for iconv.
  emulators/vmw: Replace OSVERSION checks with ICONV_LIB checks and include
  <iconv.h> instead of <sys/iconv.h>.
  irc/scrollz: Override configure test for iconv.
  japanese/chasen-base: Override configure test for iconv and patch
  configure so it no longer adds -liconv to linker flags just because it
  happens to be installed.
  japanses/eb: Patch configure test for iconv.
  japanses/eblook: Override configure test for iconv.
  java/jikes: Override configure test for iconv.
  multimedia/transcode: Bump PORTREVISION because only one plugin links with
  libiconv now.
  net/c3270: Override configure test for iconv.
  net/samba4*: Bump PORTREVISION because it no longer uses libiconv.  The
  configure script will always add -liconv to the linker flags when it
  happens to be installed which would be wrong but later on binaries are
  linked with -Wl,--as-needed and the linker discards -liconv because it
  finds iconv*() functions in libc now and no longer in libiconv.
  net-mgmt/icinga-*: Remove dependency on iconv.
  net-mgmt/netxms: Patch configure so it no longer adds -liconv to linker
  flags just because it happens to be installed.
  net/asterisk11: Patch configure so it no longer adds -liconv to linker
  flags just because it happens to be installed.
  net-p2p/transmission-*: Override configure test for iconv.
  www/htmlcxx: Override configure test for iconv.
  www/httrack: Override configure test for iconv.
  www/xapian-omega: Override configure test for iconv.
  x11/mrxvt(-devel): Add USES=iconv and override configure test for iconv.
  x11/x3270: Override configure test for iconv.
  x11-wm/jwm: Override configure test for iconv.

  PR:		202838
  Exp-run by:	antoine
  Approved by:	portmgr (antoine)

Changes:
  head/Mk/Uses/iconv.mk
  head/audio/deadbeef/Makefile
  head/audio/deadbeef/files/patch-junklib.c
  head/comms/hidapi/Makefile
  head/comms/hidapi/files/patch-configure.ac
  head/comms/hidapi/files/patch-libusb-hid.c
  head/converters/libiconv/Makefile
  head/converters/libiconv/files/patch-lib-iconv.c
  head/deskutils/fbreader/Makefile
  head/deskutils/ljclive/Makefile
  head/deskutils/owncloudclient/Makefile
  head/deskutils/owncloudclient/files/patch-cmake-modules-FindIconv.cmake
  head/devel/aegis/Makefile
  head/devel/libexplain/Makefile
  head/devel/sdl20/Makefile
  head/emulators/vmw/Makefile
  head/emulators/vmw/files/patch-vmshf.c
  head/irc/scrollz/Makefile
  head/japanese/chasen-base/Makefile
  head/japanese/eb/Makefile
  head/japanese/eb/files/patch-m4-gettext-m4
  head/japanese/eblook/Makefile
  head/java/jikes/Makefile
  head/multimedia/transcode/Makefile
  head/net/asterisk11/Makefile
  head/net/c3270/Makefile
  head/net/samba4/Makefile
  head/net/samba41/Makefile
  head/net/samba42/Makefile
  head/net-mgmt/icinga-classicweb/Makefile
  head/net-mgmt/icinga-core/Makefile.common
  head/net-mgmt/netxms/Makefile
  head/net-p2p/transmission-cli/Makefile
  head/www/htmlcxx/Makefile
  head/www/httrack/Makefile
  head/www/xapian-omega/Makefile
  head/x11/mrxvt/Makefile
  head/x11/mrxvt-devel/Makefile
  head/x11/x3270/Makefile
  head/x11-wm/jwm/Makefile
Comment 15 Olivier - interfaSys sàrl 2015-10-19 12:56:49 UTC
This change breaks a lot of port with the following configuration:

* FreeBSD 9
* gcc49 from ports
* libiconv from ports

Basically, the new make file looks for /usr/include/iconv.h which is part of 9.x last time I checked: https://svnweb.freebsd.org/base/release/9.3.0/include/iconv.h?revision=268523&view=markup

and then decides to completely ignore the libiconv port, forcing ports which need -liconv to look in base which just doesn't work.
Comment 16 Tijl Coosemans freebsd_committer freebsd_triage 2015-10-19 14:44:34 UTC
Please just remove WITH_ICONV from /etc/make.conf or /etc/src.conf and remove /usr/include/iconv.h.  Iconv support is available in FreeBSD 9 but it is disabled by default and it should not be enabled because there are several known problems with it.
Comment 17 Olivier - interfaSys sàrl 2015-10-19 19:34:44 UTC
Worked perfectly, thanks!