Bug 215882 - [exp-run] always set LANG=C and LC_ALL=C
Summary: [exp-run] always set LANG=C and LC_ALL=C
Status: Closed FIXED
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Ports Framework (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Only Me
Assignee: Tijl Coosemans
URL:
Keywords:
: 216124 (view as bug list)
Depends on:
Blocks: 212624 215792
  Show dependency treegraph
 
Reported: 2017-01-08 21:32 UTC by Tijl Coosemans
Modified: 2017-01-18 14:00 UTC (History)
3 users (show)

See Also:


Attachments
patch (67.81 KB, patch)
2017-01-08 21:32 UTC, Tijl Coosemans
no flags Details | Diff
patch2 (68.34 KB, patch)
2017-01-16 20:02 UTC, Tijl Coosemans
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Tijl Coosemans freebsd_committer 2017-01-08 21:32:55 UTC
Created attachment 178636 [details]
patch

The output of tools like awk, sort, tr,... depends on the current locale set by the user.  The attached patch adds LANG=C and LC_ALL=C at the beginning of bsd.port.mk and exports them so all commands are executed with the C locale.

It also introduces USE_LOCALE=<locale> that adds LANG=<locale> and LC_ALL=<locale> to CONFIGURE_ENV and MAKE_ENV so upstream build systems can be executed with a different locale (e.g. USE_LOCALE=en_US.UTF-8).
Comment 1 Antoine Brodin freebsd_committer 2017-01-15 21:51:22 UTC
See also bug #216124
Comment 2 Gerald Pfeifer freebsd_committer 2017-01-15 22:02:05 UTC
Of course, after I finally get to submit the patch which I had been
testing since fall, Tijl submitted this just a week earlier. :-)

This patch is quite more extensive than mine, and I intentionally only
used LC_COLLATE and in a more limited environment.

Do we really want/need to globally set LANG and the other LC_s ?
Comment 3 Tijl Coosemans freebsd_committer 2017-01-16 14:05:32 UTC
(In reply to Gerald Pfeifer from comment #2)
Yes we want the other LC_* too.  Bug 215792 requires LC_NUMERIC=C.  Use of character classes with tr(1) requires LC_CTYPE=C.  Consistent output from date(1) requires LC_TIME=C.  These are all handled at once by setting LC_ALL=C.  

LANG is used by setlocale(3) as default value for LC_* variables that aren't set, so normally it isn't used when LC_ALL is set, but I've seen patches in the ports tree that show there's code out there that looks at LANG directly so it's safer to set it as well.

I also think we need to set these globally (with .export) at the beginning of bsd.port.mk to rule out any locale problems in the ports tree itself.  The only commands not captured by this are != assignments before any inclusion of bsd.port.*mk.  I now see I forgot about bsd.port.subdir.mk.  Maybe LANG/LC_ALL should be set there as well?
Comment 4 Antoine Brodin freebsd_committer 2017-01-16 18:18:15 UTC
There is 1 new failure:

+ {"origin"=>"mail/wanderlust", "phase"=>"build", "errortype"=>"???"}

Failure log:

http://package23.nyi.freebsd.org/data/110amd64-default-PR215882/2017-01-16_06h31m26s/logs/errors/wanderlust-emacs25-2.14.0_17.log
Comment 5 Tijl Coosemans freebsd_committer 2017-01-16 20:02:29 UTC
Created attachment 178969 [details]
patch2

Attempt to fix mail/wanderlust.
Comment 6 Antoine Brodin freebsd_committer 2017-01-18 12:08:46 UTC
Exp-run looks fine
Comment 7 commit-hook freebsd_committer 2017-01-18 13:21:38 UTC
A commit references this bug:

Author: tijl
Date: Wed Jan 18 13:20:39 UTC 2017
New revision: 431796
URL: https://svnweb.freebsd.org/changeset/ports/431796

Log:
  The output of tools like awk, date, sort, tr,... depends on the current
  locale set by the user.  Add LANG=C and LC_ALL=C at the beginning of
  bsd.port.mk and export them so all commands are executed with the C locale.
  LC_ALL=C overrides all other LC_* variables.  LANG is used by setlocale(3)
  as default value for LC_* variables, so normally it isn't used when LC_ALL
  is set, but there's code out there that looks at LANG directly so it's safer
  to set it as well.  The only commands not captured by this are !=
  assignments before any inclusion of bsd.port.*mk.

  Introduce USE_LOCALE=<locale> that adds LANG=<locale> and LC_ALL=<locale> to
  CONFIGURE_ENV and MAKE_ENV so upstream build systems can be executed with a
  different locale (e.g. USE_LOCALE=en_US.UTF-8).

  PR:		215882
  Exp-run by:	antoine
  Approved by:	portmgr (antoine)

Changes:
  head/Mk/Scripts/check-stagedir.sh
  head/Mk/Uses/elixir.mk
  head/Mk/Uses/gem.mk
  head/Mk/bsd.commands.mk
  head/Mk/bsd.port.mk
  head/archivers/libzip/Makefile
  head/audio/denemo/Makefile
  head/audio/libamrnb/Makefile
  head/audio/libamrwb/Makefile
  head/base/gcc/Makefile
  head/cad/astk-serveur/Makefile
  head/cad/jspice3/files/patch-build
  head/chinese/big5width/Makefile
  head/chinese/big5width/pkg-plist
  head/devel/cutter/Makefile
  head/devel/libc++/Makefile
  head/devel/libedit/Makefile
  head/devel/libopenbsd/Makefile
  head/devel/libpci/Makefile
  head/devel/libublio/Makefile
  head/devel/powerpc64-gcc/Makefile
  head/devel/riscv64-gcc/Makefile
  head/devel/tcllib/Makefile
  head/dns/maradns/Makefile
  head/editors/emacs/Makefile
  head/editors/emacs-devel/Makefile
  head/editors/libreoffice/Makefile
  head/editors/libreoffice4/Makefile
  head/editors/openoffice-4/Makefile
  head/editors/openoffice-devel/Makefile
  head/editors/tea/Makefile
  head/editors/yudit/Makefile
  head/emulators/tme/Makefile
  head/ftp/bsdftpd-ssl/files/patch-contrib-libedit-makelist
  head/ftp/curl/Makefile
  head/ftp/ftpcopy/files/patch-src_typesize.sh
  head/ftp/pure-ftpd/Makefile
  head/games/blackjackclient/Makefile
  head/games/doomsday/Makefile
  head/games/jchessboard/Makefile
  head/games/openbor/Makefile
  head/games/quake-data/Makefile
  head/german/hunspell/Makefile
  head/graphics/Coin/Makefile
  head/graphics/jogl/Makefile
  head/graphics/libimg/Makefile
  head/japanese/skk-jisyo/Makefile
  head/java/bootstrap-openjdk/Makefile.update
  head/java/icedtea-web/Makefile
  head/java/jakarta-commons-httpclient/Makefile
  head/java/jfreechart/Makefile
  head/java/openjdk6/Makefile
  head/java/openjdk7/Makefile
  head/java/openjdk8/Makefile
  head/lang/elixir/Makefile
  head/lang/gauche/Makefile
  head/lang/gcc/Makefile
  head/lang/gcc48/Makefile
  head/lang/gcc49/Makefile
  head/lang/gcc5/Makefile
  head/lang/gcc5-devel/Makefile
  head/lang/gcc6/Makefile
  head/lang/gcc6-devel/Makefile
  head/lang/ghc/Makefile
  head/lang/ghc/bsd.cabal.mk
  head/lang/mono/Makefile
  head/lang/perl5-devel/Makefile
  head/lang/perl5.20/Makefile
  head/lang/perl5.22/Makefile
  head/lang/perl5.24/Makefile
  head/lang/rubinius/Makefile
  head/lang/ruby21/Makefile
  head/lang/ruby22/Makefile
  head/lang/ruby23/Makefile
  head/lang/ruby24/Makefile
  head/lang/tcl84/Makefile
  head/lang/tcl85/Makefile
  head/lang/tcl86/Makefile
  head/lang/tcl87/Makefile
  head/lang/yap-devel/Makefile
  head/mail/mew/files/patch-info__Makefile
  head/mail/wanderlust/files/patch-Makefile
  head/mail/wanderlust-devel/files/patch-Makefile
  head/math/geonext/Makefile
  head/math/jakarta-commons-math/Makefile
  head/math/jakarta-commons-math3/Makefile
  head/math/scilab/Makefile
  head/math/scilab-toolbox-sivp/Makefile
  head/math/scilab-toolbox-swt/Makefile
  head/multimedia/bino/Makefile
  head/multimedia/libvpx/Makefile
  head/multimedia/miro/Makefile
  head/net-mgmt/smokeping/Makefile
  head/net-p2p/mldonkey/Makefile
  head/net-p2p/vuze/Makefile
  head/news/cnews/Makefile
  head/print/pdftk/Makefile
  head/science/colt/Makefile
  head/security/i2p/Makefile
  head/security/isakmpd/Makefile
  head/security/py-dfvfs/Makefile
  head/security/py-plaso/Makefile
  head/shells/fd/Makefile
  head/sysutils/cdrdao/Makefile
  head/sysutils/pciutils/Makefile
  head/sysutils/sformat/Makefile
  head/textproc/linuxdoc-tools/Makefile
  head/textproc/msort/Makefile
  head/textproc/te-hunspell/Makefile
  head/www/jericho-html/Makefile
  head/www/tdom/Makefile
  head/www/tinymce/Makefile
  head/www/trac-tags/Makefile
  head/www/trac-wysiwyg/Makefile
  head/www/websh/Makefile
  head/www/zenphoto/Makefile
  head/x11-fonts/fontconfig/Makefile
  head/x11-toolkits/Xmt/Makefile
  head/x11-toolkits/open-motif/Makefile
  head/x11-toolkits/p5-Gtk2/Makefile
Comment 8 Tijl Coosemans freebsd_committer 2017-01-18 13:22:14 UTC
*** Bug 216124 has been marked as a duplicate of this bug. ***