Bug 214384

Summary: devel/icu: update to 58.1
Product: Ports & Packages Reporter: Jan Beich <jbeich>
Component: Individual Port(s)Assignee: freebsd-office mailing list <office>
Status: Closed FIXED    
Severity: Affects Only Me CC: office, pi, portmgr, tijl, vsasjason
Priority: --- Keywords: needs-qa, patch
Version: LatestFlags: bapt: maintainer-feedback+
jbeich: exp-run?
Hardware: Any   
OS: Any   
URL: http://site.icu-project.org/download/58
See Also: https://bugzilla.mozilla.org/show_bug.cgi?id=1316540
Bug Depends on: 205120, 214389, 214437, 214536, 214703    
Bug Blocks: 218788    
Attachments:
Description Flags
v0
none
v0.1
none
v0.2
none
v0.3
none
v0.3.1
none
test case
none
v0.4
none
v0.4 (rebased against ports r426071) none

Description Jan Beich freebsd_committer 2016-11-10 11:17:27 UTC
Firefox 52 (also ESR) to arrive around 2017-03-07 is going to require ICU 58.1. In order to keep using system ICU while also benefiting from other changes let's update.
Comment 1 Jan Beich freebsd_committer 2016-11-10 11:19:37 UTC
Created attachment 176851 [details]
v0
Comment 2 Jan Beich freebsd_committer 2016-11-10 11:22:29 UTC
For now, runtime regressions are out of scope. Let's see how much build fallout there is.

  # Add to make.conf
  OPTIONS_SET += ICU UNICODE

Note, after exp-run is finished assign back to office@, not me.
Comment 3 Jan Beich freebsd_committer 2016-11-10 11:25:18 UTC
Tijl, why in ports r422711 did you bump PORTREVISION in the following non-consumers? Are there overlinking issues not advertised via _DEPENDS?

  archivers/unarchiver
  astro/viking
  cad/freecad
  cad/gmsh-occ
  cad/gmsh
  cad/opencascade
  chinese/tin
  databases/evolution-data-server
  editors/poedit
  games/0ad
  games/openttd
  graphics/inkscape
  mail/dovecot2-antispam-plugin
  mail/dovecot2-pigeonhole
  math/rpy2
  net/zebra-server
  security/certificate-transparency
  x11-toolkits/gnustep-gui
Comment 4 Tijl Coosemans freebsd_committer 2016-11-10 11:49:20 UTC
They are part of this list:
pkg rquery '%o %B' | grep libicu  | sort -u

I don't know if that's because of overlinking or real dependencies.
Comment 5 Tijl Coosemans freebsd_committer 2016-11-10 12:03:59 UTC
Because this version removed libicule, you may need to create a port of libicu-le-hb from Harfbuzz.  And then also a second port of ICU that just builds the libiculx library (which depends on libicule).
Comment 6 Jan Beich freebsd_committer 2016-11-10 13:12:34 UTC
Created attachment 176860 [details]
v0.1

Add fix for devel/py-pyicu. Apply this patch *after* the one in bug 214389. Details below:

  $ pkg rquery '%o %B' | grep 'libicul[ex]'
  games/openttd libicule.so.57     # disabled by configure
  games/openttd libiculx.so.57     # disabled by configure
  devel/py-pyicu libicule.so.57    # disabled in 1.9.4
  devel/py3-pyicu libicule.so.57   # disabled in 1.9.4
Comment 7 commit-hook freebsd_committer 2016-11-11 11:59:56 UTC
A commit references this bug:

Author: jbeich
Date: Fri Nov 11 11:58:56 UTC 2016
New revision: 425887
URL: https://svnweb.freebsd.org/changeset/ports/425887

Log:
  chinese/tin: drop PORTREVISION as it has no effect

  PR:		214384

Changes:
  head/chinese/tin/Makefile
  head/news/tin/Makefile
Comment 8 commit-hook freebsd_committer 2016-11-11 11:59:58 UTC
A commit references this bug:

Author: jbeich
Date: Fri Nov 11 11:59:17 UTC 2016
New revision: 425888
URL: https://svnweb.freebsd.org/changeset/ports/425888

Log:
  textproc/liblrdf: drop unused dependencies

  PR:		214384
  Reported by:	readelf(1)

Changes:
  head/textproc/liblrdf/Makefile
Comment 9 commit-hook freebsd_committer 2016-11-11 12:00:00 UTC
A commit references this bug:

Author: jbeich
Date: Fri Nov 11 11:59:31 UTC 2016
New revision: 425889
URL: https://svnweb.freebsd.org/changeset/ports/425889

Log:
  math/rpy2: stop linking against math/R depends

  While here, switch back to Clang build but keep RPATH which is a quirk
  to avoid loading /usr/lib/libgcc_s.so during dlopen() race.

  PR:		214384

Changes:
  head/math/rpy2/Makefile
Comment 10 commit-hook freebsd_committer 2016-11-11 12:01:02 UTC
A commit references this bug:

Author: jbeich
Date: Fri Nov 11 11:59:57 UTC 2016
New revision: 425890
URL: https://svnweb.freebsd.org/changeset/ports/425890

Log:
  Track used or drop unused devel/icu dependency

  This tries to fix discrepancy between

    $ pkg rquery '%o %B' | awk '/libicu/ { print $1 }' | sort -u
    $ git grep -lF :devel/icu | sed 's,/Makefile$,,'

  PR:		214384
  Reported by:	tijl, stage-qa, readelf(1)

Changes:
  head/archivers/unarchiver/Makefile
  head/astro/viking/Makefile
  head/chinese/fcitx/Makefile
  head/databases/evolution-data-server/Makefile
  head/dns/bundy/Makefile
  head/editors/poedit/Makefile
  head/emulators/xcpc/Makefile
  head/games/0ad/Makefile
  head/games/openttd/Makefile
  head/net/zebra-server/Makefile
  head/net-p2p/twister/Makefile
  head/security/certificate-transparency/Makefile
  head/x11-toolkits/gnustep-gui/Makefile
Comment 11 Jan Beich freebsd_committer 2016-11-11 12:01:26 UTC
Created attachment 176894 [details]
v0.2

Rebased after r425890. More consumers bumped.
Comment 12 Jan Beich freebsd_committer 2016-11-11 19:35:05 UTC
Created attachment 176905 [details]
v0.3

(In reply to Tijl Coosemans from comment #5)
> Because this version removed libicule, you may need to create a port
> of libicu-le-hb from Harfbuzz.  And then also a second port of ICU
> that just builds the libiculx library (which depends on libicule).

Done. games/openttd maintainer maybe unhappy with even more unconditional dependencies. Alas, modernizing the port is outside of scope.
Comment 13 Jan Beich freebsd_committer 2016-11-11 20:13:28 UTC
Created attachment 176906 [details]
v0.3.1

Oops, minor style + separate pkg-descr to avoid confusion. My commit queue currently looks like this:
- (bug 214389) devel/py-pyicu: update to 1.9.4
- devel/icu: update to 58.1 + bump consumers
- devel/icu-le-hb: add new port
- devel/icu-le-hb: use GitHub to pick up a few fixes
- devel/icu-lx: add new port
- games/openttd: restore right-to-left support
Comment 14 Jan Beich freebsd_committer 2016-11-11 23:52:47 UTC
Created attachment 176908 [details]
test case

https://ssl.icu-project.org/trac/changeset/38674 crashes Firefox as ucol_getKeywordValuesForLocale() now returns bogus collations. Any ideas?

$ cc -o test test-ucol-list.c $(pkg-config icu-i18n --libs --cflags)

# ICU 58.1
$ ./test
test: (null)
test: (null)
test: (null)
test: (null)

# ICU 57.1
$ ./test
test: standard
test: emoji
test: eor
test: search
Comment 15 Jan Beich freebsd_committer 2016-11-12 00:06:18 UTC
Comment on attachment 176908 [details]
test case

Nevermind, https://ssl.icu-project.org/trac/ticket/12827
Comment 16 Jan Beich freebsd_committer 2016-11-12 00:29:53 UTC
Created attachment 176909 [details]
v0.4

Integrated upstream fix for comment 14.
Comment 17 Jan Beich freebsd_committer 2016-11-12 10:03:52 UTC
head-armv6 build is green...

Add back office@ to CC (after comment 2) and start maintainer timeout countdown. exp-run can go (and block) in parallel.
Comment 18 Jan Beich freebsd_committer 2016-11-13 23:03:10 UTC
Created attachment 176973 [details]
v0.4 (rebased against ports r426071)
Comment 19 Jan Beich freebsd_committer 2016-11-14 03:34:43 UTC
As it stands I'm accidentally testing consumers via bug 199601 but no fallout so far. ;)
Comment 20 Jan Beich freebsd_committer 2016-11-14 21:05:18 UTC
@antoine, did you apply patches bug 214389 and bug 214437 as well ?
Comment 21 Antoine Brodin freebsd_committer 2016-11-15 22:17:22 UTC
On 10.3 i386, there are 3 ports that fail to link (statically) when sqlite3 is compiled with ICU enabled.  I believe those failures are not new.

+ {"origin"=>"devel/subversion-static", "pkgname"=>"subversion-static-1.9.4", "phase"=>"build", "errortype"=>"linker_error"}
+ {"origin"=>"sysutils/cbsd", "pkgname"=>"cbsd-11.0.4_1", "phase"=>"build", "errortype"=>"linker_error"}
+ {"origin"=>"sysutils/lnav", "pkgname"=>"lnav-0.8.1", "phase"=>"build", "errortype"=>"linker_error"}

http://package23.nyi.freebsd.org/data/103i386-default-PR214384/2016-11-14_20h14m36s/logs/errors/subversion-static-1.9.4.log
http://package23.nyi.freebsd.org/data/103i386-default-PR214384/2016-11-14_20h14m36s/logs/errors/cbsd-11.0.4_1.log
http://package23.nyi.freebsd.org/data/103i386-default-PR214384/2016-11-14_20h14m36s/logs/errors/lnav-0.8.1.log

There was also chromium failing, I have to test the patch from bug #214536
Comment 22 Jan Beich freebsd_committer 2016-11-15 22:38:00 UTC
> devel/subversion-static -> linker_error
> sysutils/cbsd -> linker_error
> sysutils/lnav -> linker_error

Underspecified dependencies during static linking, not related to the update. Filed as bug 214552, bug 214551, bug 214538. Try restarting the build with

  # Add to make.conf
  databases_sqlite3_UNSET += ICU
Comment 23 Jan Beich freebsd_committer 2016-11-15 22:44:12 UTC
> #### /usr/local/etc/poudriere.d/make.conf ####
> #### /usr/local/etc/poudriere.d/PR214384-make.conf ####
> WITH=	ICU

Did you miss comment 2? _UNSET=ICU (in comment 22) won't cancel WITH=ICU.
Comment 24 Antoine Brodin freebsd_committer 2016-11-18 18:16:32 UTC
Exp-run looks fine.
Comment 25 Jan Beich freebsd_committer 2016-11-18 18:45:26 UTC
Thanks but don't forget to grant exp-run? bugzilla flag. I think, only portmgr peers can do so.
Comment 26 Antoine Brodin freebsd_committer 2016-11-18 19:46:25 UTC
(In reply to Jan Beich (mail not working) from comment #25)
I can't, I see only an empty " " or a question mark "?" in the list.
Comment 27 commit-hook freebsd_committer 2016-11-19 18:22:23 UTC
A commit references this bug:

Author: jbeich
Date: Sat Nov 19 18:21:28 UTC 2016
New revision: 426525
URL: https://svnweb.freebsd.org/changeset/ports/426525

Log:
  devel/icu: update to 58.1

  Changes:	http://site.icu-project.org/download/58
  PR:		214384
  Exp-run by:	antoine
  Approved by:	office (bapt)

Changes:
  head/archivers/unarchiver/Makefile
  head/astro/osmium/Makefile
  head/audio/musicpd/Makefile
  head/audio/ncmpcpp/Makefile
  head/databases/couchdb/Makefile
  head/databases/evolution-data-server/Makefile
  head/databases/firebird25-server/Makefile
  head/databases/postgresql92-server/Makefile
  head/databases/sqlite3/Makefile
  head/deskutils/calibre/Makefile
  head/devel/boost-libs/Makefile
  head/devel/dee/Makefile
  head/devel/glib20/Makefile
  head/devel/hs-text-icu/Makefile
  head/devel/icu/Makefile
  head/devel/icu/distinfo
  head/devel/icu/files/patch-layout_Makefile.in
  head/devel/icu/files/patch-r39484
  head/devel/icu/pkg-plist
  head/devel/objecthash/Makefile
  head/devel/pecl-intl/Makefile
  head/devel/php70-intl/Makefile
  head/devel/py-pyicu/Makefile
  head/devel/qt4-corelib/Makefile
  head/devel/qt5-core/Makefile
  head/dns/libpsl/Makefile
  head/editors/libreoffice/Makefile.common
  head/editors/libreoffice4/Makefile
  head/editors/poedit/Makefile
  head/emulators/open-vm-tools/Makefile
  head/games/0ad/Makefile
  head/games/an/Makefile
  head/games/liblcf/Makefile
  head/games/openttd/Makefile
  head/graphics/libcdr01/Makefile
  head/graphics/libfreehand/Makefile
  head/graphics/mapnik/Makefile
  head/graphics/tesseract/Makefile
  head/irc/znc/Makefile
  head/lang/gnustep-base/Makefile
  head/lang/parrot/Makefile
  head/lang/phantomjs/Makefile
  head/lang/swift/Makefile
  head/lang/v8-devel/Makefile
  head/mail/dovecot2/Makefile
  head/mail/postfix/Makefile
  head/mail/postfix-current/Makefile
  head/mail/thunderbird/Makefile
  head/math/R/Makefile
  head/misc/sword/Makefile
  head/multimedia/aegisub/Makefile
  head/multimedia/subtitlecomposer-kde4/Makefile
  head/net/yaz/Makefile
  head/net/zebra-server/Makefile
  head/news/tin/Makefile
  head/print/harfbuzz/Makefile
  head/print/libmspub01/Makefile
  head/print/tex-xetex/Makefile
  head/print/texlive-base/Makefile
  head/security/certificate-transparency/Makefile
  head/sysutils/gdisk/Makefile
  head/sysutils/osquery/Makefile
  head/sysutils/tracker/Makefile
  head/textproc/R-cran-stringi/Makefile
  head/textproc/dwdiff/Makefile
  head/textproc/ibus-qt/Makefile
  head/textproc/libe-book/Makefile
  head/textproc/libvisio01/Makefile
  head/textproc/raptor2/Makefile
  head/textproc/rubygem-charlock_holmes/Makefile
  head/textproc/zorba/Makefile
  head/www/chromium/Makefile
  head/www/cppcms/Makefile
  head/www/firefox/Makefile
  head/www/firefox-esr/Makefile
  head/www/libxul/Makefile
  head/www/node/Makefile
  head/www/node4/Makefile
  head/www/node6/Makefile
  head/www/seamonkey/Makefile
  head/www/webkit-gtk2/Makefile
  head/www/webkit-gtk3/Makefile
  head/www/webkit-qt5/Makefile
  head/www/webkit2-gtk3/Makefile
  head/x11-toolkits/gnustep-gui/Makefile
Comment 28 commit-hook freebsd_committer 2016-11-19 18:22:25 UTC
A commit references this bug:

Author: jbeich
Date: Sat Nov 19 18:21:44 UTC 2016
New revision: 426526
URL: https://svnweb.freebsd.org/changeset/ports/426526

Log:
  devel/icu-le-hb: add new port

  PR:		214384
  Exp-run by:	antoine
  Approved by:	office (bapt)

  icu-le-hb is a library implementing the ICU Layout Engine (icu-le) API
  using external HarfBuzz library for implementation. This is useful as
  a compatibility layer to make applications using ICU Layout Engine to
  use HarfBuzz without porting them to use the HarfBuzz API.

  The code is mostly a trimmed down version of icu/source/layout, with
  all shapers stripped out, and HarfBuzz integrated.

  http://www.freedesktop.org/wiki/Software/HarfBuzz/

Changes:
  head/devel/Makefile
  head/devel/icu-le-hb/
  head/devel/icu-le-hb/Makefile
  head/devel/icu-le-hb/distinfo
  head/devel/icu-le-hb/pkg-descr
  head/devel/icu-le-hb/pkg-plist
Comment 29 commit-hook freebsd_committer 2016-11-19 18:22:27 UTC
A commit references this bug:

Author: jbeich
Date: Sat Nov 19 18:21:57 UTC 2016
New revision: 426527
URL: https://svnweb.freebsd.org/changeset/ports/426527

Log:
  devel/icu-le-hb: use GitHub to pick up a few fixes

  Changes:	https://github.com/behdad/icu-le-hb/compare/1.0.3...3f0c789
  PR:		214384
  Exp-run by:	antoine
  Approved by:	office (bapt)

Changes:
  head/devel/icu-le-hb/Makefile
  head/devel/icu-le-hb/distinfo
Comment 30 commit-hook freebsd_committer 2016-11-19 18:22:29 UTC
A commit references this bug:

Author: jbeich
Date: Sat Nov 19 18:22:10 UTC 2016
New revision: 426528
URL: https://svnweb.freebsd.org/changeset/ports/426528

Log:
  devel/icu-lx: add new port

  PR:		214384
  Exp-run by:	antoine
  Approved by:	office (bapt)

  The ParagraphLayout object will analyze the text into runs of text in
  the same font, script and direction, and will create a LayoutEngine
  object for each run. The LayoutEngine will transform the characters into
  glyph codes in visual order.

  Clients can use this to break a paragraph into lines, and to display the
  glyphs in each line.

  http://site.icu-project.org/

Changes:
  head/devel/Makefile
  head/devel/icu/Makefile
  head/devel/icu-lx/
  head/devel/icu-lx/Makefile
  head/devel/icu-lx/pkg-descr
  head/devel/icu-lx/pkg-plist
Comment 31 commit-hook freebsd_committer 2016-11-19 18:23:31 UTC
A commit references this bug:

Author: jbeich
Date: Sat Nov 19 18:23:19 UTC 2016
New revision: 426529
URL: https://svnweb.freebsd.org/changeset/ports/426529

Log:
  games/openttd: restore right-to-left support after r426525

  PR:		214384
  Exp-run by:	antoine
  Approved by:	portmgr blanket

Changes:
  head/games/openttd/Makefile
Comment 32 commit-hook freebsd_committer 2016-11-20 01:24:10 UTC
A commit references this bug:

Author: jbeich
Date: Sun Nov 20 01:23:58 UTC 2016
New revision: 426548
URL: https://svnweb.freebsd.org/changeset/ports/426548

Log:
  www/firefox: handle more line break classes after r426525

  PR:		214384 (for tracking)
  Obtained from:	upstream + martin@NetBSD
  Security:	cargo cult

Changes:
  head/www/firefox/Makefile
  head/www/firefox/files/patch-bug1315986