Bug 243853 - sysutils/iocage doesn't handle locales
Summary: sysutils/iocage doesn't handle locales
Status: Closed Works As Intended
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Some People
Assignee: Michael Gmelin
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2020-02-03 20:09 UTC by Emrion
Modified: 2020-07-19 11:58 UTC (History)
2 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Emrion 2020-02-03 20:09:46 UTC
py37-iocage-1.2.4 makes errors when locale isn't "C" or "...UTF8".

-----------
$ export LANG="en_US.UTF-8"
$ sudo iocage start ftp
* Starting ftp
  + Started OK
  + Using devfs_ruleset: 1000 (iocage generated default)
  + Using IP options: ip4.addr=re0|192.168.5.12/24 ip4.saddrsel=1 ip4=new ip6.saddrsel=0 ip6=disable
  + Starting services OK
  + Executing poststart OK
$ sudo iocage stop ftp
* Stopping ftp
  + Executing prestop OK
  + Stopping services OK
  + Removing devfs_ruleset: 1000 OK
  + Removing jail process OK
  + Executing poststop OK

$ export LANG="fr_FR.ISO8859-15"                                
$ sudo iocage start ftp                                         
* Starting ftp                                                                   
  + Started OK                                                                   
  + Using devfs_ruleset: 1000 (iocage generated default)                         
  + Using IP options: ip4.addr=re0|192.168.5.12/24 ip4.saddrsel=1 ip4=new ip6.saddrsel=0 ip6=disable                                                              
Traceback (most recent call last):                                               
  File "/usr/local/bin/iocage", line 10, in <module>                             
    sys.exit(cli())                                                              
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 764, in __call__                                                                              
    return self.main(*args, **kwargs)                                            
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 717, in main 
    rv = self.invoke(ctx)                                                        
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1137, in invoke                                                                               
    return _process_result(sub_ctx.command.invoke(sub_ctx))                      
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/iocage_cli/start.py", line 60, in cli
    ioc.IOCage(jail=jail, rc=rc).start(ignore_exception=ignore)
  File "/usr/local/lib/python3.7/site-packages/iocage_lib/iocage.py", line 1907, in start
    used_ports=used_ports,
  File "/usr/local/lib/python3.7/site-packages/iocage_lib/ioc_start.py", line 87, in __init__
    raise e
  File "/usr/local/lib/python3.7/site-packages/iocage_lib/ioc_start.py", line 84, in __init__
    self.__start_jail__()
  File "/usr/local/lib/python3.7/site-packages/iocage_lib/ioc_start.py", line 769, in __start_jail__
    + exec_start, None, unjailed=True, decode=True
  File "/usr/local/lib/python3.7/site-packages/iocage_lib/ioc_exec.py", line 268, in __init__
    self.output = list(silent)
  File "/usr/local/lib/python3.7/site-packages/iocage_lib/ioc_exec.py", line 243, in exec_jail
    yield rtrn_stdout.decode(), rtrn_stderr.decode()
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe9 in position 9: invalid continuation byte
-----------
Comment 1 Emrion 2020-02-04 18:37:54 UTC
Seems I eventually found a fix.

In line 45 of /usr/local/lib/python3.7/site-packages/iocage_cli/__init__.py:
os.environ["LANG"] = ""

Do not forget to recalculate and set the checksum of the package:
pkg -r py37-iocage

So, iocage is happy and the environment variable LANG is only modified in its process. Hope there is no drawback.
Comment 2 Emrion 2020-02-28 19:11:25 UTC
Am I the only one affected by this bug?
Just upgraded py37-iocage to version 1.2.5 and it's the same.
I patched in the same way... There is one line to add, is this so difficult?

By  the way it was: pkg check -r py37-iocage to recalculate the checksum after applying this "patch". Sorry.
Comment 3 Li-Wen Hsu freebsd_committer 2020-06-16 23:23:32 UTC
Over to maintainer.
Comment 4 Michael Gmelin freebsd_committer 2020-06-19 17:39:38 UTC
(In reply to Emrion from comment #2)

In general you should use iocage with an UTF-8 locale, this is also listed as one of its requirements in its README file (section Requirements):

https://github.com/iocage/iocage/blob/4c29dc20e2e4f2f35574f05e0cf80d1ed9c0f6b2/README.md#requirements

This is also what is recommended in the FreeBSD Mastery jails book.

So basically this works as designed. As this is only about the port of iocage in the FreeBSD ports system and not about developing iocage, I intend to close the bug as WONTFIX for the moment.

----

That said, I agree that the behavior isn't nice and that the situation needs improvement. The best way you can accomplish this is by opening an issue or - even better - a pull request at the iocage project:

https://github.com/iocage/iocage

They are friendly people in general. In case your change gets accepted, I'm more than happy to bring it into the port as a patch ASAP without waiting for an official release.

When opening an issue there, link to this PR and feel free to tag me (@grembo on github), so I can follow the discussion.

By the way, I tried to reproduce the issue locally but couldn't in a very simple/basic test, so I assume that there was some special character in your output/jail that causes the decoding error (which can happen easily).

The reason that setting LANG="" works is that it makes iocage default to en_US.UTF-8 (see https://github.com/iocage/iocage/blob/4bb844c98e2348eead6894a7e38c6a2fe035188a/iocage_lib/ioc_exec.py#L72).

In general it's a bit weird that iocage_cli sets the locale always to en_US.UTF-8, while ioc_exec and iocage.py only do so in case LANG isn't set.