Bug 264760

Summary:	devel/msp430-debug-stack: Fails to fetch PDF document: slau320aj.pdf fetch: transfer timed out
Product:	Ports & Packages	Reporter:	Robert Clausecker <fuz>
Component:	Individual Port(s)	Assignee:	Lev A. Serebryakov <lev>
Status:	Closed FIXED
Severity:	Affects Many People	Keywords:	needs-patch
Priority:	---	Flags:	bugzilla: maintainer-feedback? (lev) koobs: merge-quarterly?
Version:	Latest
Hardware:	Any
OS:	Any

Description Robert Clausecker freebsd_committer

2022-06-19 01:06:43 UTC

The port tries to fetch two documents (probably chip datasheets) from TI's website.  Unfortunately these files do not fetch due to a timeout, preventing the port from being built:

=> slau320aj.pdf doesn't seem to exist in /usr/ports/distfiles/.
=> Attempting to fetch http://www.ti.com/lit/ug/slau320aj/slau320aj.pdf
fetch: transfer timed out
=> Attempting to fetch http://distcache.FreeBSD.org/ports-distfiles/slau320aj.pdf
fetch: http://distcache.FreeBSD.org/ports-distfiles/slau320aj.pdf: Not Found
=> Couldn't fetch it - please try to retrieve this
=> port manually into /usr/ports/distfiles/ and try again.
*** Error code 1

Please fix this by either (a) providing a mirror for the datasheet or (b) changing to build so the datasheet is not included by default.

Comment 1 Kubilay Kocak freebsd_committer

2022-06-19 01:16:27 UTC

Fetching the current URI (with http) scheme results in a 301 permanent redirect.

Fetching either http or https version indeed results in a timeout condition. 

However, fetching with a custom user agent appears to succeed. This may be a case of a TI a user-agent specific block.

# fetch -vv --user-agent="foo" http://www.ti.com/lit/ug/slau320aj/slau320aj.pdf
...
>>> GET /lit/ug/slau320aj/slau320aj.pdf HTTP/1.1
>>> Host: www.ti.com
>>> Accept: */*
>>> User-Agent: foo
>>> Connection: close
>>>
<<< HTTP/1.1 200 OK
<<< Server: Apache
...
<<< Last-Modified: Wed, 11 May 2022 15:58:37 GMT
<<< ETag: "1af3c3-5debe83ada73f"
...
<<< Content-Length: 1766339
<<< Cache-Control: max-age=60
...
<<< Content-Type: application/pdf
...
offset 0, length -1, size -1, clength 1766339
remote size / mtime: 1766339 / 1652284717

This may be a case of a TI a user-agent specific block.

Comment 2 commit-hook freebsd_committer

2022-06-21 09:32:17 UTC

A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/ports/commit/?id=143c06fb4384acb59497dd2745cbb92ad8051375

commit 143c06fb4384acb59497dd2745cbb92ad8051375
Author:     Lev A. Serebryakov <lev@FreeBSD.org>
AuthorDate: 2022-06-21 09:30:05 +0000
Commit:     Lev A. Serebryakov <lev@FreeBSD.org>
CommitDate: 2022-06-21 09:30:40 +0000

    devel/msp430-debug-stack: fix documentation fetching.

     TI bans "automatic" downloaders for unknwon reason, fake
    fetch's User-Agent.

    PR:             264760

 devel/msp430-debug-stack/Makefile | 3 +++
 1 file changed, 3 insertions(+)

Comment 3 Lev A. Serebryakov freebsd_committer

2022-06-21 09:33:07 UTC

Fixed, thnx!

Comment 4 Robert Clausecker freebsd_committer

2022-06-21 11:58:51 UTC

(In reply to Lev A. Serebryakov from comment #3)

This is really not a good solution.  If TI bans automatic fetching, we should respect that.  Faking a user agent to evade the ban is kind of a dick move and will only work temporarily at best.

How about you host a copy of the document on LOCAL and use that in MASTER_SITES?  Alternatively, make installing these datasheets a non-default option.

Comment 5 Lev A. Serebryakov freebsd_committer

2022-06-21 16:53:40 UTC

(In reply to Robert Clausecker from comment #4)
You are right.
I've put these files into LOCAL.

Comment 6 Robert Clausecker freebsd_committer

2022-06-21 17:52:34 UTC

(In reply to Lev A. Serebryakov from comment #5)

Thanks a lot!