Bug 82661 - [patch] Add support to filter MASTER_SITES and PATCH_SITES by protocol in bsd.port.mk
Summary: [patch] Add support to filter MASTER_SITES and PATCH_SITES by protocol in bsd...
Status: In Progress
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Ports Framework (show other bugs)
Version: Latest
Hardware: Any Any
: Normal Affects Only Me
Assignee: Port Management Team
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2005-06-26 11:00 UTC by Vasil Dimov
Modified: 2019-05-21 22:15 UTC (History)
1 user (show)

See Also:


Attachments
bsd.port.mk_fetch_only_site.diff (2.68 KB, patch)
2005-06-26 11:00 UTC, Vasil Dimov
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Vasil Dimov 2005-06-26 11:00:35 UTC
Add support to fetch only from sites whose names (inluding the protocol)
start with user-defined string. This can be used to fetch only from http
or ftp sites.

Vaida Bogdan <vaidab@Safe-mail.net> requested this feature in PR/82309
and I think it will be useful.

In the implementation user-variable FETCH_ONLY_SITES is introduced that
contains list of allowed prefixes, for example: FETCH_ONLY_SITES=ftp://
will allow fetching only from ftp sites. By the way the same can be
achieved with FETCH_ONLY_SITES=ftp

I have placed a shell function in a make variable and
define/use it two times (for distfiles and patchfiles)
to avoid the copy-paste technology :-)
Comment 1 Mathieu Arnold freebsd_committer 2005-06-26 11:29:24 UTC
Responsible Changed
From-To: freebsd-ports-bugs->portmgr

their job :-)
Comment 2 Kris Kennaway freebsd_committer 2005-10-21 00:56:43 UTC
State Changed
From-To: open->feedback

I don't want to add another complication to the already extremely 
complicated fetch support in bsd.port.mk.  Instead, I'd like to see 
the fetch stuff yanked out into separate pluggable scripts, which can 
be easily replaced with different filtering/sorting/munging 
algorithms. 

I talked to eik about this some time ago, and I think he had a partial 
implementation in devel/portmk, but it was removed because it was 
never finished (see the attic though).
Comment 3 Erwin Lansing freebsd_committer 2006-12-29 21:00:08 UTC
State Changed
From-To: feedback->suspended

Suspend until someone refactors the fetching code. 

http://www.freebsd.org/cgi/query-pr.cgi?pr=82661 

Manually adding to audit trail by linimon:

at the portmgr meeting at BSDCan 2007, we agreed that this is still a
worthwhile idea, if we can find someone to implement it.  Perhaps this
is worth revisiting after this year's SoC?

Comment 4 flz 2009-09-17 15:37:12 UTC
This is a slightly different approach than what vd suggested but also
more generic. It also re-uses what we have for RANDOMIZE_MASTER_SITES.

With this patch [1], you can retain only http sites like that:

SITES_FILTER="| tr ' ' '\n' | perl -e 'while (<>) { print "$_ " if /^http/; }'"

You can also call a script if you want to do something a bit more
complicated (like filter by TLD, ...):

SITES_FILTER="| ${PORTSDIR}/Tools/scripts/foo.pl"

[1] http://people.freebsd.org/~flz/local/ports/sites-filter.diff

-- 
Florent Thoumie
flz@FreeBSD.org
FreeBSD Committer
Comment 5 Vasil Dimov freebsd_committer 2009-09-17 17:32:50 UTC
Hello,

Florent's patch looks very simple and elegant. The only downside
is that users will have to hack ~!@#!~#~$!%$^#%& in order to e.g. get
only http:// sites. This can be worked around with a few good examples
(that can be copy-pasted) for the most common cases.

Thank you!

-- 
Vasil Dimov
moc.elcaro@vomid.lisav        Software Developer @ Oracle/Innobase Oy
gro.DSBeerF@dv                Committer @ FreeBSD.org
gro.d5v@dv                    Home @ Sofia, Bulgaria
Comment 6 flz 2009-09-17 20:11:12 UTC
On Thu, Sep 17, 2009 at 5:40 PM, Vasil Dimov <vd@freebsd.org> wrote:

>
>  Hello,
>
>  Florent's patch looks very simple and elegant. The only downside
>  is that users will have to hack ~!@#!~#~$!%$^#%& in order to e.g. get
>  only http:// sites. This can be worked around with a few good examples
>  (that can be copy-pasted) for the most common cases.
>

We could add a script in ports/Tools/scripts/ to do that, so you'll just
need to set:

SITES_FILTER="| ${PORTSDIR}/Tools/scripts/foo.pl"

-- 
Florent Thoumie
flz@FreeBSD.org
FreeBSD Committer
Comment 7 Andrew 2019-05-21 22:15:09 UTC
I can refactor the linked patch into something workable pretty easily, but there's  question of whether we actually want to do this or not.  I think there's some value in being able to "|grep -E '^https'" your master sites if you're particular about that sort of thing.  
If we think this is worthwhile, I can refactor the linked patch into something that works (and doesn't break RANDOMIZE_SITES), but it's up to portmgr@ to decide if we want this or should just close it.