Bug 231650 - lang/python36: Unreasonable error: UnicodeDecodeError: 'ascii' codec can't decode byte
Summary: lang/python36: Unreasonable error: UnicodeDecodeError: 'ascii' codec can't de...
Status: Closed Not A Bug
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Only Me
Assignee: Kubilay Kocak
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2018-09-24 02:36 UTC by Yuri Victorovich
Modified: 2018-09-24 04:11 UTC (History)
1 user (show)

See Also:
bugzilla: maintainer-feedback? (python)


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Yuri Victorovich freebsd_committer freebsd_triage 2018-09-24 02:36:43 UTC
Several ports had this problem that was patched.

Testcase: math/py-cyipopt
How to reproduce:
1. Remove the patch files/patch-ipopt_version.py
2. Run: make FLAVOR=py36
   Observe this error:
===>  Configuring for py36-cyipopt-0.1.8
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "setup.py", line 28, in <module>
    exec(open('ipopt/version.py').read())
  File "/usr/local/lib/python3.6/encodings/ascii.py", line 26, in decode
    return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 199: ordinal not in range(128)


The file setup.py is marked as utf-8. The file ipopt/version.py that it reads is also marked as utf-8. Python should be using the utf-8 codec to read this file and not fail like this.

This doesn't happen with python-2.7.
Comment 1 Kubilay Kocak freebsd_committer freebsd_triage 2018-09-24 04:11:42 UTC
This is a common class of bug not specific to FreeBSD.

Without an encoding specified for the open() method, it relies on the default system encoding(non-deterministic), which may not be UTF8 (in this case ascii) [1].

The solution is to specify the encoding to be used, so as not to be platform/system dependent.

Examples: 

 * https://bugs.launchpad.net/pbr/+bug/1704472
 * https://github.com/mozilla/unicode-slugify/issues/16
 * https://bitbucket.org/genomeinformatics/simlord/issues/1/error-when-installing-simlord-ascii-codec
 * https://github.com/morepath/morepath/issues/286
 * https://github.com/chaoss/grimoirelab-perceval/issues/32
 * https://github.com/rbarrois/python-semanticversion/issues/13

 * Many others. Google: "setup.py" "codec can't decode byte" site:.github.com

[1] https://docs.python.org/3/library/functions.html#open