Bug 269298 - sysutils/iocage: Update to 1.8, new upstream
Summary: sysutils/iocage: Update to 1.8, new upstream
Status: Closed FIXED
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Some People
Assignee: Michael Gmelin
URL:
Keywords:
Depends on: 279450
Blocks: 270016 270530
  Show dependency treegraph
 
Reported: 2023-02-03 06:19 UTC by Alexander Leidinger
Modified: 2024-11-03 13:58 UTC (History)
5 users (show)

See Also:
bugzilla: maintainer-feedback? (grembo)


Attachments
upgrade to 1.3 from new home (5.48 KB, patch)
2024-05-29 13:14 UTC, geoffroy desvernay
no flags Details | Diff
upgrade to 1.4 from new home (5.48 KB, patch)
2024-05-29 14:47 UTC, geoffroy desvernay
no flags Details | Diff
git diff sysutils/iocage to 1.5 and new home (5.48 KB, patch)
2024-05-31 12:35 UTC, geoffroy desvernay
no flags Details | Diff
git patch for sysutils/iocage 1.6 on new home (5.61 KB, patch)
2024-05-31 16:17 UTC, geoffroy desvernay
no flags Details | Diff
git patch for sysutils/iocage 1.6 on new home (6.25 KB, patch)
2024-06-03 11:11 UTC, geoffroy desvernay
no flags Details | Diff
git patch for sysutils/iocage 1.7 on new home (7.14 KB, patch)
2024-06-18 13:57 UTC, geoffroy desvernay
no flags Details | Diff
git patch for sysutils/iocage 1.7 on new home (8.73 KB, patch)
2024-07-24 12:13 UTC, geoffroy desvernay
no flags Details | Diff
git diff for 1.8 on new home (7.53 KB, patch)
2024-09-20 07:13 UTC, geoffroy desvernay
no flags Details | Diff
git patch for sysutils/iocage 1.8 on new home (18.24 KB, patch)
2024-09-20 07:31 UTC, geoffroy desvernay
no flags Details | Diff
git patch for sysutils/iocage 1.8 on new home (8.88 KB, patch)
2024-09-20 07:34 UTC, geoffroy desvernay
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Alexander Leidinger freebsd_committer freebsd_triage 2023-02-03 06:19:01 UTC
As per https://github.com/iocage/iocage/issues/1289#issuecomment-1268638684 the iocage project is abandoned.

As per https://github.com/iocage/iocage/pull/1292#issuecomment-1368531213 a replacement project is also not moving forward.

Suggestion: add a deprecation notice to the port.
Comment 1 geoffroy desvernay 2024-05-29 13:14:06 UTC
Created attachment 251061 [details]
upgrade to 1.3 from new home

iocage is not dead :)

I took maintainership on new home https://github.com/freebsd/iocage

Here is a 1.3 version with all needed patches included

Tested working here on our production servers
Comment 2 Michael Gmelin freebsd_committer freebsd_triage 2024-05-29 13:16:41 UTC
(In reply to geoffroy desvernay from comment #1)

Good to hear.

Will be interesting to diff this to the current version and see if it works as expected (I lost a bit track between the changes to the port, things applied upstream and some local patches that were not ready for public consumption).
Comment 3 geoffroy desvernay 2024-05-29 14:47:07 UTC
Created attachment 251065 [details]
upgrade to 1.4 from new home

fix list -h command
Comment 4 geoffroy desvernay 2024-05-29 14:48:28 UTC
I took upstream as-is (with many unversionned patches), and checked all patches used in port's version
Comment 5 geoffroy desvernay 2024-05-31 12:35:01 UTC
Created attachment 251114 [details]
git diff sysutils/iocage to 1.5 and new home

additional changes:
  - allow use of allow.nfsd jail parameter
Comment 6 geoffroy desvernay 2024-05-31 16:17:37 UTC
Created attachment 251118 [details]
git patch for sysutils/iocage 1.6 on new home

version 1.6 fixes empty HTTP_PROXY bug (making phttpget fail)
Comment 7 Alexander Leidinger freebsd_committer freebsd_triage 2024-06-01 11:45:15 UTC
This depends on py-libzfs and on -current py-libzfs needs the patch in Bug 279450.
Comment 8 geoffroy desvernay 2024-06-02 17:35:40 UTC
I think it shouldn't block here as iocage never use zpool_add
Comment 9 Alexander Leidinger freebsd_committer freebsd_triage 2024-06-02 20:42:16 UTC
Are the iocage patches tested against python 3.11? The current port spews out this:
---snip---
...
  File "/usr/local/lib/python3.11/site-packages/iocage_cli/start.py", line 57, in cli
    ioc.IOCage(rc=rc, silent=True).start(ignore_exception=ignore)
  File "/usr/local/lib/python3.11/site-packages/iocage_lib/iocage.py", line 1872, in start
    self.__jail_order__("start", ignore_exception=ignore_exception)
  File "/usr/local/lib/python3.11/site-packages/iocage_lib/iocage.py", line 208, in __jail_order__
    self.__rc__(boot_order, action, ignore_exception)
  File "/usr/local/lib/python3.11/site-packages/iocage_lib/iocage.py", line 256, in __rc__
    err, msg = self.start(j, ignore_exception=ignore_exception)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/iocage_lib/iocage.py", line 1912, in start
    ioc_start.IOCStart(
  File "/usr/local/lib/python3.11/site-packages/iocage_lib/ioc_start.py", line 87, in __init__
    raise e
  File "/usr/local/lib/python3.11/site-packages/iocage_lib/ioc_start.py", line 84, in __init__
    self.__start_jail__()
  File "/usr/local/lib/python3.11/site-packages/iocage_lib/ioc_start.py", line 903, in __start_jail__
    rctl_jail.remove_rctl_rules()
  File "/usr/local/lib/python3.11/site-packages/iocage_lib/ioc_json.py", line 306, in remove_rctl_rules
    iocage_lib.ioc_exec.SilentExec(
  File "/usr/local/lib/python3.11/site-packages/iocage_lib/ioc_exec.py", line 268, in __init__
    self.output = list(silent)
                  ^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/iocage_lib/ioc_exec.py", line 259, in exec_jail
    raise iocage_lib.ioc_exceptions.CommandFailed(
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/iocage_lib/ioc_exceptions.py", line 34, in __init__
    collections.Iterable
AttributeError: module 'collections' has no attribute 'Iterable'
---snip---
Comment 10 Alexander Leidinger freebsd_committer freebsd_triage 2024-06-02 20:44:03 UTC
(In reply to geoffroy desvernay from comment #8)
Something needs to be done with py-libzfs, as it is a build dependency, and without any patch to it on -current, iocage will not be available as a package on -current.
Comment 11 Alexander Leidinger freebsd_committer freebsd_triage 2024-06-03 09:08:01 UTC
I tried this patch, something is missing:
---snip---
# iocage list         
Traceback (most recent call last):
  File "/usr/local/bin/iocage", line 6, in <module>
    from iocage_cli import cli
  File "/usr/local/lib/python3.11/site-packages/iocage_cli/__init__.py", line 38, in <module>
    import iocage_lib.ioc_check as ioc_check
  File "/usr/local/lib/python3.11/site-packages/iocage_lib/ioc_check.py", line 30, in <module>
    import iocage_lib.ioc_common
  File "/usr/local/lib/python3.11/site-packages/iocage_lib/ioc_common.py", line 47, in <module>
    import iocage_lib.ioc_exec
  File "/usr/local/lib/python3.11/site-packages/iocage_lib/ioc_exec.py", line 28, in <module>
    import iocage_lib.ioc_json
  File "/usr/local/lib/python3.11/site-packages/iocage_lib/ioc_json.py", line 39, in <module>
    import iocage_lib.ioc_create
  File "/usr/local/lib/python3.11/site-packages/iocage_lib/ioc_create.py", line 35, in <module>
    import iocage_lib.ioc_fstab
  File "/usr/local/lib/python3.11/site-packages/iocage_lib/ioc_fstab.py", line 41, in <module>
    from iocage_lib.utils import load_ctypes_library, ensure_unicode_str
  File "/usr/local/lib/python3.11/site-packages/iocage_lib/utils.py", line 1, in <module>
    import six
ModuleNotFoundError: No module named 'six'
---snip---
Comment 12 geoffroy desvernay 2024-06-03 11:11:10 UTC
Created attachment 251184 [details]
git patch for sysutils/iocage 1.6 on new home

add missing py-six dependency
Comment 13 Alexander Leidinger freebsd_committer freebsd_triage 2024-06-03 11:49:47 UTC
iocage list (comment #11). I haven't tried restarting / starting (comment #9).
Comment 14 Alexander Leidinger freebsd_committer freebsd_triage 2024-06-03 11:53:38 UTC
The previous comment was supposed to be "iocage list works now".
Comment 15 Alexander Leidinger freebsd_committer freebsd_triage 2024-06-06 08:00:56 UTC
Version 1.6 has issues with python 3.11 on start like version 1.2 has:
---snip---
  + Executing poststart OK
Traceback (most recent call last):
  File "/usr/local/bin/iocage", line 10, in <module>
    sys.exit(cli())
             ^^^^^
  File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1434, in invoke    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/iocage_cli/start.py", line 60, in cli
    ioc.IOCage(jail=jail, rc=rc).start(ignore_exception=ignore)
  File "/usr/local/lib/python3.11/site-packages/iocage_lib/iocage.py", line 1823, in start
    ioc_start.IOCStart(
  File "/usr/local/lib/python3.11/site-packages/iocage_lib/ioc_start.py", line 87, in __init__
    raise e
  File "/usr/local/lib/python3.11/site-packages/iocage_lib/ioc_start.py", line 84, in __init__
    self.__start_jail__()
  File "/usr/local/lib/python3.11/site-packages/iocage_lib/ioc_start.py", line 975, in __start_jail__
    rctl_jail.remove_rctl_rules()
  File "/usr/local/lib/python3.11/site-packages/iocage_lib/ioc_json.py", line 310, in remove_rctl_rules
    iocage_lib.ioc_exec.SilentExec(
  File "/usr/local/lib/python3.11/site-packages/iocage_lib/ioc_exec.py", line 274, in __init__
    self.output = list(silent)                  ^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/iocage_lib/ioc_exec.py", line 265, in exec_jail
    raise iocage_lib.ioc_exceptions.CommandFailed(
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/iocage_lib/ioc_exceptions.py", line 34, in __init__
    collections.Iterable
AttributeError: module 'collections' has no attribute 'Iterable'
---snip---
Comment 16 Alexander Leidinger freebsd_committer freebsd_triage 2024-06-09 12:59:46 UTC
Here is a fix for the issue:
    https://stackoverflow.com/questions/59809785/i-get-a-attributeerror-module-collections-has-no-attribute-iterable-when-i


short:
in "/usr/local/lib/python3.11/site-packages/iocage_lib/ioc_exceptions.py"
replace "import collections"
by "import collections.abc as collections"

Works for me.
Comment 17 geoffroy desvernay 2024-06-18 13:11:03 UTC
ok, fixed here: https://github.com/freebsd/iocage/issues/11

I'll upload a new patch soon
Comment 18 geoffroy desvernay 2024-06-18 13:12:52 UTC
for libzfs, it's only used for tests, I'll remove the dependency first, as patching this in a retro-compatible way is not on my skills list …
Comment 19 geoffroy desvernay 2024-06-18 13:57:41 UTC
Created attachment 251537 [details]
git patch for sysutils/iocage 1.7 on new home

collections Iterable fix, linsysfs support and less dependencies

libzfs is not a RUN_ dependency anymore (should unblock with broken libzfs)
Comment 20 geoffroy desvernay 2024-07-24 12:13:39 UTC
Created attachment 252256 [details]
git patch for sysutils/iocage 1.7 on new home

rebase patch
Comment 21 Gleb Popov freebsd_committer freebsd_triage 2024-09-13 17:48:50 UTC
(In reply to Michael Gmelin from comment #2)
Michael, I think this is ready to get committed. Are you OK with updating the port to 1.7?
Comment 22 Michael Gmelin freebsd_committer freebsd_triage 2024-09-13 17:56:22 UTC
(In reply to Gleb Popov from comment #21)

It seems like the dust settled a bit around this. I would like to do some local testing first though, let's aim for having this landed before 2024Q4 is cut, ok?
Comment 23 Gleb Popov freebsd_committer freebsd_triage 2024-09-13 17:58:22 UTC
If you say so. Geoffroy did a lot of testing on his side and I also tried out the new version via iocage-devel. But if you need to do testing yourself, sure.
Comment 24 geoffroy desvernay 2024-09-20 07:13:36 UTC
Created attachment 253680 [details]
git diff for 1.8 on new home

Here with ~150 jails on ~15 servers we use it on production for a long time.
Release 1.7 is, in my view, a clean replacement for old and now broken 1.2.

I just released 1.8 with some bugfixes and one default behaviour change, but for new installs only (vnet_default_interface, that did break host network by default on some configs)
Comment 25 geoffroy desvernay 2024-09-20 07:31:04 UTC
Created attachment 253681 [details]
git patch for sysutils/iocage 1.8 on new home

git format-patch with commit included
Comment 26 geoffroy desvernay 2024-09-20 07:34:44 UTC
Created attachment 253682 [details]
git patch for sysutils/iocage 1.8 on new home

sorry, last one did include the patch itself :/
Comment 27 commit-hook freebsd_committer freebsd_triage 2024-09-30 20:14:55 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/ports/commit/?id=5787776b7075cb63a5a49345e2d0a37b31a271b4

commit 5787776b7075cb63a5a49345e2d0a37b31a271b4
Author:     Michael Gmelin <grembo@FreeBSD.org>
AuthorDate: 2024-09-30 20:07:44 +0000
Commit:     Michael Gmelin <grembo@FreeBSD.org>
CommitDate: 2024-09-30 20:13:57 +0000

    sysutils/iocage: Update to 1.8, new upstream

    PR:             269298
    Reported by:    netchild
    Co-authored-by: Geoffroy Desvernay <dgeo@centrale-med.fr>

 sysutils/iocage/Makefile                           | 39 +++++++---------------
 sysutils/iocage/distinfo                           | 16 ++-------
 .../files/patch-iocage__lib_ioc__common.py (gone)  | 11 ------
 .../patch-iocage__lib_ioc__exceptions.py (gone)    | 11 ------
 .../files/patch-iocage__lib_ioc__json.py (gone)    | 11 ------
 sysutils/iocage/files/patch-setup.py (gone)        | 11 ------
 6 files changed, 15 insertions(+), 84 deletions(-)