Created attachment 271630 [details] patch */*: Use C.UTF-8 locale ports 02f27a83b44d fixed the ports tree to the C locale so it would work correctly in environments where users use a locale that isn't compatible with the C locale, like xx_YY.UTF-8 locales for example where [A-Z] includes more than just upper case letters. USE_LOCALE was introduced to let ports use another locale if they needed it. The C.UTF-8 locale was created later on to be mostly compatible with the C locale. Switch the ports tree locale to C.UTF-8 and remove USE_LOCALE. The world has switched to Unicode so all ports either require it or know how to build correctly in a Unicode environment. games/aisleriot: Install file with non-ASCII name The file is extracted correctly now that the ports tree uses UTF-8. shells/fd: Pass LANG via MAKE_ARGS The upstream build system uses LANG to determine the location of Japanese manpages. Pass this through MAKE_ARGS instead of USE_LOCALE because "ja" is not a valid locale. While here use the upstream make variable BUILD for DESTDIR.
At least devel/schilybase still requires an ISO 8859-1 locale for parts of the build (translating PO files in that locale to MO files).
(In reply to Robert Clausecker from comment #1) Seems to build fine here. PO files specify the encoding in their header and gettext knows how to convert them.
(In reply to Tijl Coosemans from comment #2) It builds fine, but does the wrong thing due to the incorrect encoding. I had a lengthy discussion with upstream about this issue. I pass --no-convert to disable Unicode mode (which doesn't work for the MO file shipped in this package). I had in the past also set the locale, but that was removed at some point, I don't recall the specific circumstances. Given that it's again broken (see bug #257905), I think the port may actually need the locale setting after all.
In any case, I don't get why you'd want to remove support infrastructure for locales. Are you that sure that it'll never be relevant again?
(In reply to Robert Clausecker from comment #3) It looks like the trick to install share/locale/C/LC_MESSAGES/SCHILY_utils.mo stopped working with gettext 0.23. Gettext treats C.<encoding> as untranslated now. But you're not using gettext for translation. You're relying on the fact that gettext calls iconv on translation strings. So, instead of using gettext you could use iconv directly. However, if I were you I would remove all the gettext stuff and replace _("Jörg Schilling") and _("Joerg Schilling") with "J\xc3\xb6rg Schilling". Users that still use ISO8859-1 will see that as "Jörg Schilling", but that group is shrinking. The solution with iconv looks something like this: use iconv in main() to convert the string "J\xf6rg Schilling" from ISO8859-1 to the encoding of the current locale into some buffer, and then replace _("Jörg Schilling") and _("Joerg Schilling") everywhere with a pointer to that buffer. char jorgschilling[64] = {}; /* zero filled */ static const char jorgschilling_src[] = "J\xf6rg Schilling"; main() { ... setlocale(LC_ALL, ""); ... iconv_t icd = iconv_open("", "ISO8859-1"); char *dst = jorgschilling; size_t dstlen = sizeof(jorgschilling) - 1; char *src = jorgschilling_src; size_t srclen = sizeof(jorgschilling_src) - 1; iconv(icd, &src, &srclen, &dst, &dstlen); iconv_close(icd); ... } foo() { printf("%s\n", jorgschilling); }
(In reply to Tijl Coosemans from comment #5) Please understand that the string "Jörg Schilling" exists as a test case for a planned localisation effort. So removing gettext() in favour of an iconv() call misses the point of why it's there. The locale of the entire project is ISO 8859-1 and cannot be changed easily due to the project doing direct string manipulation all over the place. Just changing the translation strings to UTF-8 while leaving the rest in ISO 8859-1 is a recipe for disaster, as we then get mixed-encoding source files. > Gettext treats C.<encoding> as untranslated now. Yeah, that kind of sucks. I'm not sure what the maintainer was thinking there, as this also breaks cases where projects use strings that shouldn't go directly to the user for the key (e.g. to disambiguate messages that are equal in en.US but distinct in other languages). I guess I'll have to talk to the gettext maintainers once again. In any case, please understand my objection as not being about this project in particular, but about it being a really bad idea to remove locale support just because you find it fashionable to do so.
(In reply to Robert Clausecker from comment #6) The text in comment 0 is a copy-paste of my commit log. If the exp-run shows that USE_LOCALE is still needed I intend to keep it, but the problem in devel/schilybase can't be solved with USE_LOCALE.
New failures logs: https://pkg-status.freebsd.org/gohan06/data/150amd64-default-foo/2026-06-12_07h36m15s/logs/errors/doomsday-2.3.1_8.log https://pkg-status.freebsd.org/gohan06/data/150amd64-default-foo/2026-06-12_07h36m15s/logs/errors/xblast-2.10.4_20.log https://pkg-status.freebsd.org/gohan06/data/150amd64-default-foo/2026-06-12_07h36m15s/logs/errors/ja-skk-jawiki-2026.04.01.141931.log https://pkg-status.freebsd.org/gohan06/data/150amd64-default-foo/2026-06-12_07h36m15s/logs/errors/ja-skk-jisyo-cdb-20260411.log https://pkg-status.freebsd.org/gohan06/data/150amd64-default-foo/2026-06-12_07h36m15s/logs/errors/ja-today-2.12_2.log https://pkg-status.freebsd.org/gohan06/data/150amd64-default-foo/2026-06-12_07h36m15s/logs/errors/linuxdoc-tools-0.9.83_1.log
Created attachment 271742 [details] patch2 games/doomsday: Don't remove Unicode BOM games/xblast: Fix renaming of a file with a ISO8859-1 encoded "î" japanese/skk-*: Run awk with LC_ALL=C This is what the upstream Makefile does as well. It makes awk pass through 8-bit characters instead of trying to interpret them as UTF-8 in an environment with a UTF-8 locale. japanese/today: Run tr with eucJP codeset textproc/linuxdoc-tools: Generate UTF-8 LyX file
japanese/today: Do you want to remove tr since line breaks can be converted with nkf? - nkf -Se $$i | ${TR} -d '\015' > $$i.tmp ;\ - ${MV} -f $$i.tmp $$i ; \ + nkf -Sed --overwrite $$i ;\
New failure log: https://pkg-status.freebsd.org/gohan06/data/150amd64-default-foo/2026-06-29_06h42m54s/logs/errors/opencbm-0.4.99.104_2.log comms/opencbm-plugin-xa1541 was skipped because of it
Created attachment 272307 [details] patch3 Uses/electron.mk: Move changes to Uses/npm.mk following ports ed6a84c1c979. japanese/today: Apply comment 10. textproc/linuxdoc-tools: Fix locale used by expand(1) to fix problem with comms/opencbm. I tested japanese/today, textproc/linuxdoc-tools and comms/opencbm* with poudriere, so I think it's safe to commit this. If you still want to run an exp-run you have to delete the old linuxdoc-tools package.
if you tested no need for another exp-run
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/ports/commit/?id=b6ec06f079e1c883d32c788c0285c6d492eeb1c0 commit b6ec06f079e1c883d32c788c0285c6d492eeb1c0 Author: Tijl Coosemans <tijl@FreeBSD.org> AuthorDate: 2026-06-07 16:50:32 +0000 Commit: Tijl Coosemans <tijl@FreeBSD.org> CommitDate: 2026-06-30 20:03:57 +0000 */*: Use C.UTF-8 locale ports 02f27a83b44d fixed the ports tree to the C locale so it would work correctly in environments where users use a locale that isn't compatible with the C locale, e.g. xx_YY.UTF-8 where [A-Z] includes more than just upper case letters. USE_LOCALE was introduced to let ports use another locale if they needed it. The C.UTF-8 locale was created later on to be mostly compatible with the C locale. Switch the ports tree locale to C.UTF-8 and remove USE_LOCALE. The world has moved to Unicode so all ports either require it or know how to build correctly in a Unicode environment. PR: 295945 Exp-run by: antoine Mk/Scripts/dialog4ports.sh | 4 ++-- Mk/Uses/elixir.mk | 4 +--- Mk/Uses/gem.mk | 3 --- Mk/Uses/gnome.mk | 4 ---- Mk/Uses/kde.mk | 2 -- Mk/Uses/meson.mk | 3 --- Mk/Uses/npm.mk | 9 +++------ Mk/Uses/qt.mk | 5 ----- Mk/bsd.licenses.mk | 10 +++++----- Mk/bsd.port.mk | 14 ++++---------- Tools/scripts/add-port-to-category-makefile.sh | 2 +- Tools/scripts/git-diff-ports.sh | 2 +- Tools/scripts/git-get-latest-remote-version.sh | 2 +- Tools/scripts/hackage-get-latest-version.sh | 2 +- Tools/scripts/npmjs-fetch-with-dependencies.sh | 2 +- Tools/scripts/npmjs-get-latest-version.sh | 2 +- Tools/scripts/pypi-get-latest-version.sh | 2 +- audio/flac/Makefile | 4 ---- audio/flacon/Makefile | 3 +-- audio/picard/Makefile | 1 - audio/synthpod-lv2/Makefile | 2 +- biology/gatk/Makefile | 2 +- biology/jalview/Makefile | 2 -- deskutils/R-cran-exams/Makefile | 3 +-- deskutils/joplin-desktop/Makefile | 6 ++---- devel/R-cran-data.table/Makefile | 4 ---- devel/R-cran-lintr/Makefile | 2 -- devel/R-cran-sessioninfo/Makefile | 2 -- devel/R-cran-testthat/Makefile | 2 -- devel/R-cran-vctrs/Makefile | 2 -- devel/aarch64-none-elf-gcc/Makefile | 9 --------- devel/cmake-doc/Makefile | 1 - devel/cmake-man/Makefile | 1 - devel/hs-cabal-plan/Makefile | 2 -- devel/hs-git-annex/Makefile | 1 - devel/hs-hadolint/Makefile | 2 -- devel/hs-haskell-language-server/Makefile | 2 -- devel/hs-hlint/Makefile | 2 -- devel/hs-ormolu/Makefile | 2 -- devel/ice37/Makefile | 2 +- devel/meson-python/Makefile | 2 -- devel/py-appdirs/Makefile | 1 - devel/py-buildbot/Makefile | 2 +- devel/py-configparser/Makefile | 1 - devel/py-cookiecutter/Makefile | 9 ++------- devel/py-jupyter_console/Makefile | 1 - devel/py-molecule/Makefile | 4 ---- devel/py-pytest-plus/Makefile | 2 +- devel/py-python-magic/Makefile | 2 +- devel/py-qt6-qscintilla2/Makefile | 1 - devel/py-wheel/Makefile | 2 -- devel/py-wheel044/Makefile | 2 -- devel/qt6-base/Makefile | 1 - devel/radicle/Makefile | 2 +- devel/tcllib/Makefile | 1 - dns/libpsl/Makefile | 1 - editors/kakoune/Makefile | 1 - editors/libreoffice/Makefile | 1 - editors/tea/Makefile | 1 - emulators/elliott-803/Makefile | 1 - filesystems/py-dfvfs/Makefile | 4 ---- finance/fava/Makefile | 2 -- finance/hs-hledger-ui/Makefile | 1 - finance/hs-hledger-web/Makefile | 1 - finance/hs-hledger/Makefile | 1 - french/aster/Makefile | 1 - games/blackjackclient/Makefile | 3 +-- games/devd-controller-rules/Makefile | 1 - games/doomsday/Makefile | 3 --- games/jchessboard/Makefile | 1 - games/luanti/Makefile | 2 +- graphics/jogl/Makefile | 1 - graphics/oidn/Makefile | 2 +- graphics/poppler-qt5/Makefile | 3 --- graphics/rawtherapee/Makefile | 2 +- japanese/fcitx5-hazkey/Makefile | 3 +-- japanese/font-myrica/Makefile | 3 --- japanese/font-myricam/Makefile | 3 --- java/icedtea-web/Makefile | 3 --- java/jfreechart/Makefile | 1 - java/openjdk11/Makefile | 4 +--- java/openjdk17/Makefile | 4 +--- java/openjdk21/Makefile | 4 +--- korean/hunspell/Makefile | 2 +- lang/bun/Makefile | 6 +++--- lang/elixir-devel/Makefile | 1 - lang/elixir/Makefile | 1 - lang/gcc12/Makefile | 9 --------- lang/ghc/Makefile | 1 - lang/go/bsd.go.mk | 2 -- lang/kefir/Makefile | 1 - lang/purescript/Makefile | 2 -- math/apache-commons-math/Makefile | 1 - math/hs-Agda/Makefile | 3 --- math/py-matplotlib-scalebar/Makefile | 1 - math/scilab-toolbox-swt/Makefile | 1 - math/scilab/Makefile | 1 - math/ttmath/Makefile | 2 +- misc/gemini-cli/Makefile | 2 +- misc/py-spdx-tools/Makefile | 2 +- multimedia/ffmpegthumbnailer/Makefile | 2 -- multimedia/py-pymediainfo/Makefile | 3 --- net/gerbera/Makefile | 2 +- net/py-ldap0/Makefile | 1 - net/py-urllib3/Makefile | 3 +-- net/rabbitmq/Makefile | 1 - ports-mgmt/porttree/Makefile | 2 +- security/openvpn-devel/Makefile | 2 +- security/openvpn/Makefile | 2 +- security/py-maec/Makefile | 1 - security/py-plaso/Makefile | 3 --- security/py-pyopenssl/Makefile | 3 +-- security/xca/Makefile | 1 - shells/bash-completion/Makefile | 4 ---- sysutils/ansible/Makefile | 2 +- sysutils/kleene-daemon/Makefile | 2 -- sysutils/openvox-agent8/Makefile | 4 ++-- sysutils/puppet8/Makefile | 4 ++-- sysutils/py-ansible-compat/Makefile | 2 +- sysutils/py-ansible-lint/Makefile | 2 +- sysutils/py-salt/Makefile | 3 --- textproc/R-cran-hunspell/Makefile | 2 -- textproc/docbookrx/Makefile | 3 +-- textproc/ibus-typing-booster/Makefile | 1 - textproc/pageedit/Makefile | 2 -- textproc/py-pikepdf/Makefile | 2 -- textproc/py-sphinxcontrib-programoutput/Makefile | 1 - textproc/te-hunspell/Makefile | 2 +- textproc/translate-shell/Makefile | 1 - textproc/write-good/Makefile | 4 ++-- textproc/xlnt/Makefile | 2 +- www/chromium/Makefile | 2 +- www/hs-hjsmin/Makefile | 3 --- www/jericho-html/Makefile | 3 +-- www/newsraft/Makefile | 1 - www/onlyoffice-documentserver/Makefile | 2 -- www/py-django42/Makefile | 3 --- www/py-django52/Makefile | 3 --- www/py-django60/Makefile | 3 --- www/py-python-digitalocean/Makefile | 1 - www/qutebrowser/Makefile | 2 -- www/ungoogled-chromium/Makefile | 2 +- x11/foot/Makefile | 2 +- x11/qimageblitz/Makefile | 2 +- 144 files changed, 73 insertions(+), 285 deletions(-)
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/ports/commit/?id=42c8c728288121eddab5e0389ce25a95cba59669 commit 42c8c728288121eddab5e0389ce25a95cba59669 Author: Tijl Coosemans <tijl@FreeBSD.org> AuthorDate: 2026-06-10 15:57:47 +0000 Commit: Tijl Coosemans <tijl@FreeBSD.org> CommitDate: 2026-06-30 20:03:56 +0000 japanese/skk-*: Run awk with LC_ALL=C This is what the upstream Makefile does as well. It makes awk pass through 8-bit characters instead of trying to interpret them as UTF-8 when the ports tree locale becomes C.UTF-8. PR: 295945 Exp-run by: antoine japanese/skk-jawiki/Makefile | 2 +- japanese/skk-jisyo/Makefile | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-)
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/ports/commit/?id=ddda2c59e6aa72f86e9fa13ec9d8d4548f72b020 commit ddda2c59e6aa72f86e9fa13ec9d8d4548f72b020 Author: Tijl Coosemans <tijl@FreeBSD.org> AuthorDate: 2026-06-10 19:00:00 +0000 Commit: Tijl Coosemans <tijl@FreeBSD.org> CommitDate: 2026-06-30 20:03:56 +0000 japanese/today: Simplify line break conversion nkf can do line break conversion, no need for tr. The data is eucJP encoded and tr treats it as UTF-8 when the ports tree locale becomes C.UTF-8. Submitted by: mce PR: 295945 Exp-run by: antoine japanese/today/Makefile | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/ports/commit/?id=d87a1e8780312b8e6cbfbe7b7bdc6a99315c202b commit d87a1e8780312b8e6cbfbe7b7bdc6a99315c202b Author: Tijl Coosemans <tijl@FreeBSD.org> AuthorDate: 2026-06-06 11:13:41 +0000 Commit: Tijl Coosemans <tijl@FreeBSD.org> CommitDate: 2026-06-30 20:03:56 +0000 shells/fd: Pass LANG via MAKE_ARGS The upstream build system uses LANG to determine the location of Japanese manpages. Pass this through MAKE_ARGS instead of USE_LOCALE because "ja" is not a valid locale. While here use the upstream make variable BUILD for DESTDIR. PR: 295945 Exp-run by: antoine shells/fd/Makefile | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-)
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/ports/commit/?id=72b9da5ac99d1240e717857998556c4a2825d136 commit 72b9da5ac99d1240e717857998556c4a2825d136 Author: Tijl Coosemans <tijl@FreeBSD.org> AuthorDate: 2026-06-09 17:40:49 +0000 Commit: Tijl Coosemans <tijl@FreeBSD.org> CommitDate: 2026-06-30 20:03:56 +0000 textproc/linuxdoc-tools: Fix expand(1) locale Add a patch so linuxdoc runs expand(1) with the right locale to interpret multibyte characters. PR: 295945 Exp-run by: antoine textproc/linuxdoc-tools/Makefile | 6 +++++- .../files/patch-perl5lib_LinuxDocTools.pm (new) | 24 ++++++++++++++++++++++ 2 files changed, 29 insertions(+), 1 deletion(-)
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/ports/commit/?id=084afac04e9789ae9972883cd74eb920c05c8470 commit 084afac04e9789ae9972883cd74eb920c05c8470 Author: Tijl Coosemans <tijl@FreeBSD.org> AuthorDate: 2026-06-07 15:47:34 +0000 Commit: Tijl Coosemans <tijl@FreeBSD.org> CommitDate: 2026-06-30 20:03:56 +0000 games/aisleriot: Install file with non-ASCII name The file is extracted correctly now that the ports tree uses UTF-8 to extract files. PR: 295945 Exp-run by: antoine games/aisleriot/Makefile | 1 + games/aisleriot/files/patch-cards_meson.build (gone) | 10 ---------- games/aisleriot/pkg-plist | 1 + 3 files changed, 2 insertions(+), 10 deletions(-)
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/ports/commit/?id=d26c01fde278a05fcc2c9c8f865055db361083e3 commit d26c01fde278a05fcc2c9c8f865055db361083e3 Author: Tijl Coosemans <tijl@FreeBSD.org> AuthorDate: 2026-06-12 10:07:26 +0000 Commit: Tijl Coosemans <tijl@FreeBSD.org> CommitDate: 2026-06-30 20:03:57 +0000 games/xblast: Fix renaming of a file with a ISO8859-1 encoded "î" PR: 295945 Exp-run by: antoine games/xblast/Makefile | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/ports/commit/?id=f17a2a47a96dad2b82a79540b9546cd04f5ea52c commit f17a2a47a96dad2b82a79540b9546cd04f5ea52c Author: Tijl Coosemans <tijl@FreeBSD.org> AuthorDate: 2026-07-01 07:24:43 +0000 Commit: Tijl Coosemans <tijl@FreeBSD.org> CommitDate: 2026-07-01 07:37:04 +0000 textproc/linuxdoc-tools: Remove no-op post-stage This was a remnant from a previous attempt to fix the port with UTF-8. PR: 295945 textproc/linuxdoc-tools/Makefile | 4 ---- 1 file changed, 4 deletions(-)