Bug 231965 - [PowerPC64] Cross compiling powerpc64 from amd64 results in nonfunctional locale installations
Summary: [PowerPC64] Cross compiling powerpc64 from amd64 results in nonfunctional loc...
Status: Closed FIXED
Alias: None
Product: Base System
Classification: Unclassified
Component: misc (show other bugs)
Version: CURRENT
Hardware: powerpc Any
: --- Affects Only Me
Assignee: Yuri Pankov
URL: https://reviews.freebsd.org/D17603
Keywords:
: 222871 241570 (view as bug list)
Depends on:
Blocks:
 
Reported: 2018-10-04 23:47 UTC by Sean Bruno
Modified: 2020-12-20 03:52 UTC (History)
4 users (show)

See Also:
yuripv: mfc-stable12+


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Sean Bruno freebsd_committer freebsd_triage 2018-10-04 23:47:59 UTC
On the FreeBSD.ORG hosts, pylon, archon and ref12-ppc64, the installation of locales doesn't seem to be functional as reported by the use of tmux:

sbruno@ref12-ppc64:~ % tmux
tmux: need UTF-8 locale (LC_CTYPE) but have US-ASCII

I'm unsure what we need to look at here to resolve this.  It is reported that a native build of powerpc64 on a powerpc64 host results in a working locale installation.
Comment 1 Sean Bruno freebsd_committer freebsd_triage 2018-10-17 17:21:33 UTC
Now that ref12-ppc64 is a bit more stable, you can login and see this problem.  I don't know what's wrong here as the locale bits are installed in /usr/share/locale.
Comment 2 Yuri Pankov freebsd_committer freebsd_triage 2018-10-17 20:10:05 UTC
Looking at this, I wonder if powerpc ISOs (at least, 12-ALPHA10 snapshot) cross-compiled on amd64 as the locale data files are matching the ones on my amd64 system.

In any case, it looks like the check in lib/libc/locale/endian.h needs to include __powerpc__ as well, or just check for big endian.  I've cross-compiled libc with the following change on amd64, and was able to run simple test program just doing setlocale(), and even run tmux preloading the resulting libc.so.7:

diff --git a/lib/libc/locale/endian.h b/lib/libc/locale/endian.h
index d3b822788688..ade03bdc8997 100644
--- a/lib/libc/locale/endian.h
+++ b/lib/libc/locale/endian.h
@@ -45,7 +45,7 @@
  * correct macros here.
  */

-#if BYTE_ORDER == BIG_ENDIAN && defined(__mips__)
+#if BYTE_ORDER == BIG_ENDIAN
 #define        BSWAP(x)        le32toh(x)
 #else
 #define        BSWAP(x)        x

I'm NOT sure if this is correct for all cases, and if it will not break native powerpc builds, need to investigate a bit more.
Comment 3 Sean Bruno freebsd_committer freebsd_triage 2018-10-18 19:33:57 UTC
(In reply to Yuri Pankov from comment #2)
This 100% fixes the issue reported here.  You can now run tmux on ref12-ppc64 after I applied this update to libc.

Do you want to try and get this into the tree?
Comment 4 Yuri Pankov freebsd_committer freebsd_triage 2018-10-18 19:35:13 UTC
No, that was just a hack to understand the problem.  I'm working on a real fix and it's nearly ready.
Comment 5 Sean Bruno freebsd_committer freebsd_triage 2018-10-18 19:50:08 UTC
(In reply to Yuri Pankov from comment #4)
Awesome, and thank you.
Comment 6 commit-hook freebsd_committer freebsd_triage 2018-10-20 20:51:32 UTC
A commit references this bug:

Author: yuripv
Date: Sat Oct 20 20:51:08 UTC 2018
New revision: 339489
URL: https://svnweb.freebsd.org/changeset/base/339489

Log:
  Add -b/-l options to localedef(1) to specify output endianness and use
  it appropriately when building share/ctypedef and share/colldef.

  This makes the resulting locale data in EL->EB (amd64->powerpc64) cross
  build and in the native EB build match.  Revert the changes done to libc
  in r308170 as they are no longer needed.

  PR:		231965
  Reviewed by:	bapt, emaste, sbruno, 0mp
  Approved by:	kib (mentor)
  Differential Revision:	https://reviews.freebsd.org/D17603

Changes:
  head/lib/libc/locale/collate.c
  head/lib/libc/locale/endian.h
  head/lib/libc/locale/rune.c
  head/share/colldef/Makefile
  head/share/ctypedef/Makefile
  head/share/mk/bsd.endian.mk
  head/targets/pseudo/userland/Makefile.depend
  head/tools/tools/locale/tools/cldr2def.pl
  head/usr.bin/localedef/collate.c
  head/usr.bin/localedef/ctype.c
  head/usr.bin/localedef/localedef.1
  head/usr.bin/localedef/localedef.c
  head/usr.bin/localedef/localedef.h
Comment 7 Sean Bruno freebsd_committer freebsd_triage 2018-10-23 00:46:31 UTC
Just tagging this as an MFC candidate for stable/12
Comment 8 commit-hook freebsd_committer freebsd_triage 2018-10-27 21:17:26 UTC
A commit references this bug:

Author: yuripv
Date: Sat Oct 27 21:17:04 UTC 2018
New revision: 339825
URL: https://svnweb.freebsd.org/changeset/base/339825

Log:
  MFC r339489:

  Add -b/-l options to localedef(1) to specify output endianness and use
  it appropriately when building share/ctypedef and share/colldef.

  This makes the resulting locale data in EL->EB (amd64->powerpc64) cross
  build and in the native EB build match.  Revert the changes done to libc
  in r308170 as they are no longer needed.

  PR:             231965
  Reviewed by:    bapt, emaste, sbruno, 0mp
  Approved by:    re (gjb), kib (mentor)
  Differential Revision:  https://reviews.freebsd.org/D17603

Changes:
_U  stable/12/
  stable/12/lib/libc/locale/collate.c
  stable/12/lib/libc/locale/endian.h
  stable/12/lib/libc/locale/rune.c
  stable/12/share/colldef/Makefile
  stable/12/share/ctypedef/Makefile
  stable/12/share/mk/bsd.endian.mk
  stable/12/targets/pseudo/userland/Makefile.depend
  stable/12/tools/tools/locale/tools/cldr2def.pl
  stable/12/usr.bin/localedef/collate.c
  stable/12/usr.bin/localedef/ctype.c
  stable/12/usr.bin/localedef/localedef.1
  stable/12/usr.bin/localedef/localedef.c
  stable/12/usr.bin/localedef/localedef.h
Comment 9 Yuri Pankov freebsd_committer freebsd_triage 2019-09-01 06:40:51 UTC
*** Bug 222871 has been marked as a duplicate of this bug. ***
Comment 10 Yuri Pankov freebsd_committer freebsd_triage 2020-12-20 03:52:15 UTC
*** Bug 241570 has been marked as a duplicate of this bug. ***