Bug 204394

Summary: usr.bin/tr: ASCII collation passthrough doesn't work on Unicode-aware locales
Product: Base System Reporter: O. Hartmann <ohartmann>
Component: binAssignee: Baptiste Daroussin <bapt>
Status: Closed Works As Intended    
Severity: Affects Many People CC: bapt, emaste, jbeich, marino, yuripv
Priority: --- Keywords: regression
Version: 11.0-STABLEFlags: jbeich: maintainer-feedback+
Hardware: Any   
OS: Any   

Description O. Hartmann 2015-11-09 10:41:37 UTC
Port fails to compile/configure on most recent CURRENT (FreeBSD 11.0-CURRENT #0 r290570: Mon Nov  9 06:31:05 CET 2015 amd64) as shown below.

 
[...]
configure:27436: c++ -c -O2 -pipe -O3 -march=native -O3 -DLIBICONV_PLUG -fstack-protector -fno-strict-aliasing  -DLIBICONV_PLUG -fno-exceptions -fno-strict-aliasing -fno-rtti -ffunction-sections -fdata-sections -fno-exceptions -fno-math-errno -std=gnu++0x -Qunused-arguments -isystem/usr/local/include -DLIBICONV_PLUG  conftest.C 1>&5
configure:27453: checking for _Unwind_Backtrace
configure:27490: c++ -o conftest -O2 -pipe -O3 -march=native -O3 -DLIBICONV_PLUG -fstack-protector -fno-strict-aliasing  -DLIBICONV_PLUG -fno-exceptions -fno-strict-aliasing -fno-rtti -ffunction-sections -fdata-sections -fno-exceptions -fno-math-errno -std=gnu++0x -Qunused-arguments -isystem/usr/local/include -DLIBICONV_PLUG  -pthread  -L/usr/local/lib -Wl,-rpath,/usr/local/lib/firefox -fstack-protector -Wl,-z,noexecstack -Wl,-z,text conftest.C  1>&5
configure: error: --enable-chrome-format must be set to either jar, flat, or omni
*** Fix above errors and then restart with\
               "gmake -f client.mk build"
/usr/ports/www/firefox/work/mozilla-release/client.mk:364: recipe for target 'configure' failed
gmake[3]: *** [configure] Error 1
gmake[3]: Leaving directory '/usr/ports/www/firefox'
===>  Script "configure" failed unexpectedly.
Please report the problem to gecko@FreeBSD.org [maintainer] and attach the
"/usr/ports/www/firefox/work/mozilla-release/config.log" including the output
of the failure of your make command. Also, it might be a good idea to provide
an overview of all packages installed on your system (e.g. a
/usr/local/sbin/pkg-static info -g -Ea).
*** Error code 1
Comment 1 O. Hartmann 2015-11-17 05:46:54 UTC
See also Bug 204612.
Comment 2 Jan Beich freebsd_committer freebsd_triage 2015-11-19 11:30:08 UTC
Try building with LC_COLLATE=C. Recent locales update in base r290494 may break a few ports e.g.,
https://lists.freebsd.org/pipermail/freebsd-ports/2015-November/101034.html
Comment 3 O. Hartmann 2015-11-19 18:29:08 UTC
Setting LC_COLLATE=C worked! Thank you very much. It also is applicable for  Bug 204612. Same help. Same salvation.

oh
Comment 4 Jan Beich freebsd_committer freebsd_triage 2015-11-19 19:34:42 UTC
*** Bug 204612 has been marked as a duplicate of this bug. ***
Comment 5 Jan Beich freebsd_committer freebsd_triage 2015-11-19 22:02:39 UTC
Cutting down configure I've found

  # NLS nuisances.
  # Only set these to C if already set.  These must not be set unconditionally
  # because not all systems understand e.g. LANG=C (notably SCO).
  # Fixing LC_MESSAGES prevents Solaris sh from translating var values in `set'!
  # Non-C LC_CTYPE values break the ctype check.
  if test "${LANG+set}"   = set; then LANG=C;   export LANG;   fi
  if test "${LC_ALL+set}" = set; then LC_ALL=C; export LC_ALL; fi
  if test "${LC_MESSAGES+set}" = set; then LC_MESSAGES=C; export LC_MESSAGES; fi
  if test "${LC_CTYPE+set}"    = set; then LC_CTYPE=C;    export LC_CTYPE;    fi

which breaks tr(1) on a mixed locale (LANG unset)

  $ env -i LC_COLLATE=en_US.UTF-8 LC_CTYPE=C sh
  $ echo FooBar | tr A-Z a-z
  aq
  $ echo FooBar | tr '[:upper:]' '[:lower:]'
  foobar

GNU tr (coreutils) works fine but doesn't support UTF-8 at all.
Comment 6 Yuri Pankov 2018-01-08 14:04:45 UTC
http://pubs.opengroup.org/onlinepubs/9699919799/utilities/tr.html says:

c-c
...
In locales other than the POSIX locale, this construct has unspecified behavior.

So, while unfortunate, no one should rely on "c-c" constructs unless they explicitly set the POSIX("C") locale.
Comment 7 Baptiste Daroussin freebsd_committer freebsd_triage 2019-11-27 11:29:34 UTC
As yuri stated, this is an unfortunate but expected problem