Bug 218348 - ports-mgmt/portmaster: --local-packagedir --packages-local does not work with pkgng
Summary: ports-mgmt/portmaster: --local-packagedir --packages-local does not work with...
Status: Closed FIXED
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Many People
Assignee: Stefan Eßer
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2017-04-04 10:37 UTC by Ivan Rozhuk
Modified: 2017-12-19 23:08 UTC (History)
4 users (show)

See Also:
bugzilla: maintainer-feedback? (tz)


Attachments
pkgng support (10.35 KB, patch)
2017-04-04 10:37 UTC, Ivan Rozhuk
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Ivan Rozhuk 2017-04-04 10:37:39 UTC
Created attachment 181450 [details]
pkgng support

add support pkgng: --local-packagedir --packages-local now works


Scenario

On "build master":
mount nfs share to /usr/ports rw (or share it via nfs)
=== once ===
mkdir -p /usr/ports/packages/All
mkdir -p /var/cache/portmaster-download
ln -s /var/cache/portmaster-download /usr/ports/packages/
mv /usr/ports/distfiles /var/cache/
ln -s /var/cache/distfiles /usr/ports/
pkg create -a -n -o /usr/ports/packages/All
============
portmaster -Bgvda -y --delete-build-only --packages


On "slaves":
=== once ===
mkdir -p /var/cache/portmaster-download
mv /usr/ports/distfiles /var/cache/
============
mount nfs share to /usr/ports ro
portmaster -Bvda -y --delete-build-only --packages --local-packagedir=/usr/ports/packages --packages-local

"Slaves" will update installed ports from local packages, if no package found then it will build on "slave".
Many profits for llvm, firefox, libreoffice and other heavy ports distributing.
Comment 1 Ivan Rozhuk 2017-05-09 02:41:03 UTC
Torsten Zuehlsdorff, can you check and commit this patch?
Comment 2 Torsten Zuehlsdorff freebsd_committer freebsd_triage 2017-05-29 14:34:56 UTC
Thanks for the patch! I will have a look at it! :)
Comment 3 Ivan Rozhuk 2017-08-20 21:22:19 UTC
Any progress?
Comment 4 Torsten Zuehlsdorff freebsd_committer freebsd_triage 2017-12-08 14:09:09 UTC
Hand over PR to new Maintainer of portmaster :)
Comment 5 Ivan Rozhuk 2017-12-08 22:45:38 UTC
How many years wait before it will be committed? %)
Comment 6 Stefan Eßer freebsd_committer freebsd_triage 2017-12-09 07:16:23 UTC
Well, it seems that you have been waiting for less than a year by now ...

Innfact, I had considered ripping out all PACKAGES support. This is a feature that was useful with the old ports framework, when it did not provide a fully functional package upgrade functionality. Packages support adds quite some code to portmaster and makes it even harder to maintain.

But you are the second person that finds packages support in portmaster useful, so I'm leaving packages support in and will consider your patch for inclusion.

My primary target for now is adding FLAVOR support to portmaster, since it won't be usable in the future without.

Then I plan to go through the open PRs and triage and apply them if they are still useful.

Your patch will need to be heavily reworked, since support for the old ports framework has been ripped out of the master version of portmaster on GitHub.
This will simplify the patch, but it will still add another 200 odd lines of code to the already way to long portmaster script.

I really think that your use-case is better served by using poudriere, which is easy to setup based on the information in the FreeBSD handbook or Woki (I forgot, where the simple walk-through is given, which lets you start using poudriere in 10 minutes).

For that reason, I still consider reducing portmaster to port building and package creation, but without support for installation and upgrade from packages. This will need to be discussed, though.
Comment 7 John Hein 2017-12-11 03:21:38 UTC
For flavor support including parsing @<flavor> entries in MOVED, test the following change:

https://github.com/jhgit/portmaster/commit/4ef492af3f30ba1a2a2434fb8241cf1a17bca157

I'll issue a pull request on github after testing this some more (and probably a bug here as well instead of hijacking this bug).  Please provide feedback if you try it.
Comment 8 Ivan Rozhuk 2017-12-11 11:15:50 UTC
(In reply to Stefan Esser from comment #6)

I do not like poudriere because I dont use and remove from system zfs and jail.
I just want upgrade ports on my main desktop, package it into nfs share and update using this packages (if package not found then build it from sources) on other workstations.
Comment 9 Stefan Eßer freebsd_committer freebsd_triage 2017-12-11 12:00:06 UTC
You do not need ZFS for poudriere and JAIL support is always compiled into the kernel; the WITHOUT_JAIL option just controls building and installing of the jail, jexec and jls commands, which are rather small binaries (less than 100 KB total disk space).

BTW: -PP has been broken in portmaster for a long time and does not appear to have been missed by anybody but you ... (at least I did not see any other request to re-introduce it).
Comment 10 Ivan Rozhuk 2017-12-18 13:23:24 UTC
(In reply to John Hein from comment #7)
Today I update to portmaster-3.17.11_7, it does not work with packages dir on "slave" update.
Is portmaster-3.17.11_7 includes your patch?
Comment 11 Walter Schwarzenfeld freebsd_triage 2017-12-18 14:20:12 UTC
The patch applies not anymore, as of the changes in the last versions (as expected).
Comment 12 Stefan Eßer freebsd_committer freebsd_triage 2017-12-19 10:45:58 UTC
Yes, that was to be expected. I tried to avoid changes that just cause churn, but what makes your patch no longer apply is the removal of pre PKG_NG support, which was over-due. It complicated the code for no good reason, and it prevented implementation of a number of functions required to support flavors (which use pkg functionality not found in the old package system).

As explained before, I'm really not convinced that package support in portmaster is generally useful. Mixing of ports and packages is probelmatic and you are way better off using poudirere to build a consistent set of packages to install. (And this does not require ZFS, and WITHOUT_JAIL offers only a negligible reduction in make world time and disk space required).

I'd rather remove all support for packages from portmaster, than spend effort on adding support for local packages to the current version.

If there were any other users that want that feature, I might reconsider that decision, but I do not see any requests in the mail lists or bug database.

You are of course free to update your patch to apply to the current portmaster version, but I plan to refactor and rework large parts of portmaster and it will then again need work (if it can be made to work at all).
Comment 13 Stefan Eßer freebsd_committer freebsd_triage 2017-12-19 12:30:42 UTC
OK, I give in ...

Please check the latest portmaster version (3.17.11_9), which has the still relevant parts of the patch hand-applied.

Since PKG_NG support has been removed before, a large part of the patch was no longer relevant.

I have no use for this feature and have only tested, that it does not cause any regression in portmaster, but not whether it performs any useful action.

BTW: It would have been much easier to apply the patch, had it been created against the portmaster program and not the patch file of the port.
Comment 14 Ivan Rozhuk 2017-12-19 23:08:17 UTC
Thanks!
3.17.11_9 - works ok.