Bug 204135 - ports-mgmt/portscout: not finding new distfiles
Summary: ports-mgmt/portscout: not finding new distfiles
Status: Closed FIXED
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Only Me
Assignee: Ryan Steinmetz
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2015-10-29 22:47 UTC by Don Lewis
Modified: 2016-03-02 20:03 UTC (History)
0 users

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Don Lewis freebsd_committer freebsd_triage 2015-10-29 22:47:40 UTC
I maintain a number of ports where:
   MASTER_SITES=   FARSIGHT LOCAL/truckman/farsight
At one time, portscout would notify me when new distfiles showed up, but this hasn't worked with my ports for quite a while.  This might have broken when I started mirroring the distfiles in ~/public_distfiles and added LOCAL to MASTER_SITES, but I'm not sure.

Since I don't copy new distfiles to ~/public_distfiles until I know about them, and since there is no index page for directories under LOCAL, I tried restricting
portscout to the primary master site by adding
  PORTSCOUT=${MASTER_SITES:N*FreeBSD*:S/^/site:/}
to net/sie-nmsg/Makefile, which evaluates to:
  %make -V PORTSCOUT
  site:https://dl.farsightsecurity.com/dist/sie-nmsg/
If I view that URL in my browser, I see that this directory contains 
sie-nmsg-1.2.0.tar.gz, but portscount does not see it.  On this page
<http://portscout.freebsd.org/truckman@freebsd.org.html>, the entry for this port contains:
sie-nmsg	net	1.1.0		2015-10-29 02:21 GMT	2015-10-29 10:27 GMT	

Also, the "M" column in this table contains "G" for some of the ports, while the other ports have blank entries in the column.  There does not seem to be any pattern to which ports have the "G" flag.
Comment 1 Don Lewis freebsd_committer freebsd_triage 2015-11-04 23:54:39 UTC
I changed the definition of PORTSCOUT to this;

PORTSCOUT=      ${MASTER_SITES:N*FreeBSD*:S/^https/site:http/}

which evaluates to:

%make -V PORTSCOUT
site:http://dl.farsightsecurity.com/dist/sie-nmsg/


Still no joy.  My best guess is that portscount can't parse the nginx-generated index page.
Comment 2 Don Lewis freebsd_committer freebsd_triage 2015-11-05 01:04:49 UTC
Actually, it does appear to be an https issue after all.  When this site is accessed with an http URL, it returns a 301 redirect to the corresponding https URL.

I installed portscount locally and added some debug printfs.  When I printed $response->status_line after the $ua->get($site) call, I observed the following:
  501 Protocol scheme 'https' is not supported (LWP::Protocol::https not installed
)


The solution seems to be as simple as installing the p5-LWP-Protocol-https
package.

# portscout showupdates
portscout v0.8.1, by Shaun Amott

SQLite is currently only supported in non-forking mode!
--> Forcing num_children => 0...

truckman@freebsd.org's ports:
  net/sie-nmsg 1.1.0 -> 1.2.0



I also needed to install p5-DBD-SQLite to get portscout to run at all.