Bug 228360

Summary: ports-mgmt/poudriere: option to fetch distfiles at once?
Product: Ports & Packages Reporter: Michael Danilov <mike.d.ft402>
Component: Individual Port(s)Assignee: Bryan Drewery <bdrewery>
Status: New ---    
Severity: Affects Only Me CC: vsasjason, w.schwarzenfeld
Priority: --- Flags: bugzilla: maintainer-feedback? (bdrewery)
Version: Latest   
Hardware: Any   
OS: Any   
Attachments:
Description Flags
Make list of fetched ports.
none
Fetch distfiles in parallel. none

Description Michael Danilov 2018-05-19 13:07:36 UTC 
There seems to be no option to fetch all needed distfiles at once. This would be very useful on intermittent connection or when one wants to limit Internet access to certain times.

Simply running "make fetch recursive" in Ports before running Poudriere is very slow because on each port it starts anew instead of making a list of what to build.

I tried making a custom script that uses "make missing" of Ports and it works most of the time, except that it reguarly forgets about some distfiles, something like Linux compat and I still see them downloaded by Poudriere later.

Another problem is, downloading the distfiles is not parallel and CPU is almost idle.
Comment 1 Michael Danilov 2018-05-19 13:14:28 UTC 
Created attachment 193530 [details]
Make list of fetched ports.

Attaching the ugly scripts I'm using to give the idea.
Comment 2 Michael Danilov 2018-05-19 13:15:07 UTC 
Created attachment 193531 [details]
Fetch distfiles in parallel.
Comment 3 Michael Danilov 2018-05-19 13:16:08 UTC 
(unparallelised download is also inefficient because it does not use the Internet bandwidth fully -- would it be better to run more fetch jobs than CPUs?)
Comment 4 Walter Schwarzenfeld freebsd_triage 2019-08-14 23:11:20 UTC 
Maintainer feedback, please!