Bug 246670 - bsdtar: Fails to extract (UTF-8) under QEMU_EMULATING
Summary: bsdtar: Fails to extract (UTF-8) under QEMU_EMULATING
Status: New
Alias: None
Product: Base System
Classification: Unclassified
Component: bin (show other bugs)
Version: Unspecified
Hardware: arm64 Any
: --- Affects Some People
Assignee: freebsd-bugs mailing list
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2020-05-23 01:26 UTC by Danilo G. Baio
Modified: 2020-05-23 02:44 UTC (History)
3 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Danilo G. Baio freebsd_committer 2020-05-23 01:26:20 UTC
root@12-aarch64-default:~ # locale
LANG=
LC_CTYPE="C"
LC_COLLATE="C"
LC_TIME="C"
LC_NUMERIC="C"
LC_MONETARY="C"
LC_MESSAGES="C"
LC_ALL=

root@12-aarch64-default:/tmp # tar -zxf /portdistfiles/python/Sphinx-3.0.3.tar.gz
tar: Pathname can't be converted from UTF-8 to current locale.
tar: Error exit delayed from previous errors.
root@12-aarch64-default:/tmp # echo $?
1

root@12-aarch64-default:/tmp # setenv LANG en_US.UTF-8
root@12-aarch64-default:/tmp # setenv LC_ALL en_US.UTF-8
root@12-aarch64-default:/tmp # setenv MM_CHARSET UTF-8
root@12-aarch64-default:/tmp # locale
LANG=en_US.UTF-8
LC_CTYPE="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_ALL=en_US.UTF-8

root@12-aarch64-default:/tmp # tar -zxf /portdistfiles/python/Sphinx-3.0.3.tar.gz
root@12-aarch64-default:/tmp # echo $?
0


More details on bug #246618
Reported by jbeich@
Comment 1 Conrad Meyer freebsd_committer 2020-05-23 01:40:05 UTC
What's the bug here?  In the C locale, some UTF-8 characters are not representable.  If you're actually using UTF-8, you should set your locale correctly.  In 13-CURRENT this is somewhat improved by making the default locale "C.UTF-8" instead of "C[.USASCII]".
Comment 2 Danilo G. Baio freebsd_committer 2020-05-23 01:55:01 UTC
(In reply to Conrad Meyer from comment #1)

It just happens under QEMU_EMULATING, default system.
Comment 3 Conrad Meyer freebsd_committer 2020-05-23 01:57:02 UTC
Seems like a problem with QEMU_EMULATING.
Comment 4 Danilo G. Baio freebsd_committer 2020-05-23 02:01:50 UTC
(In reply to Conrad Meyer from comment #3)

Right. Should I move it to ports (emulators/qemu-user-static) ?
Comment 5 Jan Beich freebsd_committer 2020-05-23 02:44:42 UTC
(In reply to Conrad Meyer from comment #1)
> In the C locale, some UTF-8 characters are not representable.

Indeed. Why bsdtar(1) only errors out when built statically? QEMU_EMULATING builds use native-xtools which are native binaries built statically in order to speed up emulated builds.

--- tar tf (en_US.UTF-8)
+++ tar tf (C)
@@ -1258,7 +1258,7 @@
 Sphinx-3.0.3/tests/roots/test-images/subdir/svgimg.pdf
 Sphinx-3.0.3/tests/roots/test-images/subdir/svgimg.svg
 Sphinx-3.0.3/tests/roots/test-images/subdir/svgimg.xx.svg
-Sphinx-3.0.3/tests/roots/test-images/testimäge.png
+Sphinx-3.0.3/tests/roots/test-images/testim?ge.png
 Sphinx-3.0.3/tests/roots/test-index_on_title/
 Sphinx-3.0.3/tests/roots/test-index_on_title/conf.py
 Sphinx-3.0.3/tests/roots/test-index_on_title/contents.rst