Bug 239114

Summary: Many FreeBSD mirrors are outdated or broken
Product: Services Reporter: Gautam Mani <execve>
Component: FTP/WWW Sites & MirrorsAssignee: FreeBSD Mirror Admin <mirror-admin>
Status: In Progress ---    
Severity: Affects Some People CC: grahamperrin, lwhsu, mirror-admin, philip
Priority: ---    
Version: unspecified   
Hardware: Any   
OS: Any   
See Also: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=270775
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=270774
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=270773
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=270772
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=270771
Attachments:
Description Flags
log file
none
Mirrors to be checked
none
Program used to check that mirrors are working.
none
mirrors.csv none

Description Gautam Mani 2019-07-10 17:02:50 UTC 
Created attachment 205664 [details]
log file

I stumbled upon the fact that many of the FreeBSD mirrors are broken or un-available. For some, a host lookup also fails. 

I made a small program to check each of the FTP and HTTP mirrors. I collected the list from the list of mirrors available in the handbook. I did not attempt to download each file - but just checked for existance of one file - in this case, 12.0 dvd1-iso for amd64. rsync has not been checked.

The program and the csv file (list of mirrors) is attached just in case.
The log file has a full trace of each of the errors for the servers.

I thought of writing to the admin emails myself, but given that the number is non-significant - I think it better that this is somehow taken up from the FreeBSD services team. 

Given that this is one of the first things that a new user experiences, it is important that we get these mirrors fixed or else update the documentation accordingly. 

I am hoping that there are no false alarms thrown by the program - I checked quite a few cases myself manually to validate the same and it looks OK except for one character encoding case.

Do let me know how I can help further with this.

Below is the list of mirrors that need to be checked.

ALERT:Mirror successful(ftp://ftp5.FreeBSD.org/pub/FreeBSD/), 550 releases/ISO-IMAGES/12.0/: No such file or directory.: False
ALERT:Mirror successful(ftp://ftp1.am.freebsd.org/pub/FreeBSD/), timed out: False
ALERT:Mirror successful(ftp://ftp.au.FreeBSD.org/pub/FreeBSD/), timed out: False
ALERT:Mirror successful(ftp://ftp2.au.FreeBSD.org/pub/FreeBSD/), 550 /pub/FreeBSD/: No such file or directory: False
ALERT:Mirror successful(ftp://ftp2.br.FreeBSD.org/FreeBSD/), 'ascii' codec can't encode characters in position 22-23: ordinal not in range(128): False
ALERT:Mirror successful(ftp://ftp3.br.FreeBSD.org/pub/FreeBSD/), 550 FreeBSD-12.0-RELEASE-amd64-dvd1.iso: No such file or directory: False
ALERT:Mirror successful(ftp://ftp4.br.FreeBSD.org/pub/FreeBSD/), timed out: False
ALERT:Mirror successful(ftp://ftp.cn.freebsd.org/pub/FreeBSD/), timed out: False
ALERT:Mirror successful(ftp://ftp2.cz.FreeBSD.org/pub/FreeBSD/), 550 Failed to change directory.: False
ALERT:Mirror successful(ftp://ftp.ee.FreeBSD.org/pub/FreeBSD/), timed out: False
ALERT:Mirror successful(ftp://ftp6.fr.FreeBSD.org/pub/FreeBSD/), timed out: False
ALERT:Mirror successful(http://ftp4.de.FreeBSD.org/pub/FreeBSD/), : False
ALERT:Mirror successful(ftp://ftp7.de.FreeBSD.org/pub/FreeBSD/), 550 releases/ISO-IMAGES/12.0/: Datei oder Verzeichnis nicht gefunden: False
ALERT:Mirror successful(http://ftp7.de.FreeBSD.org/pub/FreeBSD/), : False
ALERT:Mirror successful(ftp://ftp8.de.FreeBSD.org/pub/FreeBSD/), 550 Failed to change directory.: False
ALERT:Mirror successful(ftp://ftp2.kr.FreeBSD.org/pub/FreeBSD/), 550 Failed to change directory.: False
ALERT:Mirror successful(ftp://ftp.lt.FreeBSD.org/pub/FreeBSD/), timed out: False
ALERT:Mirror successful(http://ftp.lt.FreeBSD.org/pub/FreeBSD/), HTTPConnectionPool(host='ftp.lt.freebsd.org', port=80): Max retries exceeded with url: /pub/FreeBSD/releases/ISO-IMAGES/12.0/FreeBSD-12.0-RELEASE-amd64-dvd1.iso (Caused by ConnectTimeoutError(<urllib3.connection.HTTPConnection object at 0x8022464a8>, 'Connection to ftp.lt.freebsd.org timed out. (connect timeout=15)')): False
ALERT:Mirror successful(ftp://ftp.nz.FreeBSD.org/pub/FreeBSD/), 550 Failed to change directory.: False
ALERT:Mirror successful(http://ftp.nz.FreeBSD.org/pub/FreeBSD/), : False
ALERT:Mirror successful(http://ftp.ru.FreeBSD.org/FreeBSD/), : False
ALERT:Mirror successful(ftp://ftp.isu.net.sa/pub/ftp.freebsd.org/), 550 Failed to change directory.: False
ALERT:Mirror successful(ftp://ftp.za.FreeBSD.org/pub/FreeBSD/), 550 Can't change directory to /pub/FreeBSD/: No such file or directory: False
ALERT:Mirror successful(ftp://ftp.es.FreeBSD.org/pub/FreeBSD/), timed out: False
ALERT:Mirror successful(http://ftp.es.FreeBSD.org/pub/FreeBSD/), ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response',)): False
ALERT:Mirror successful(ftp://ftp3.es.FreeBSD.org/pub/FreeBSD/), timed out: False
ALERT:Mirror successful(ftp://ftp2.se.FreeBSD.org/pub/FreeBSD/), timed out: False
ALERT:Mirror successful(ftp://ftp3.se.FreeBSD.org/pub/FreeBSD/), timed out: False
ALERT:Mirror successful(ftp://ftp6.se.FreeBSD.org/pub/FreeBSD/), timed out: False
ALERT:Mirror successful(http://ftp2.tw.FreeBSD.org/pub/FreeBSD/), HTTPConnectionPool(host='ftp2.tw.freebsd.org', port=80): Max retries exceeded with url: /pub/FreeBSD/releases/ISO-IMAGES/12.0/FreeBSD-12.0-RELEASE-amd64-dvd1.iso (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x8022645c0>: Failed to establish a new connection: [Errno 61] Connection refused',)): False
ALERT:Mirror successful(ftp://ftp6.tw.FreeBSD.org/pub/FreeBSD/), timed out: False
ALERT:Mirror successful(ftp://ftp11.tw.FreeBSD.org/pub/FreeBSD/), timed out: False
ALERT:Mirror successful(http://ftp11.tw.FreeBSD.org/FreeBSD/), HTTPConnectionPool(host='ftp11.tw.freebsd.org', port=80): Max retries exceeded with url: /FreeBSD/releases/ISO-IMAGES/12.0/FreeBSD-12.0-RELEASE-amd64-dvd1.iso (Caused by ConnectTimeoutError(<urllib3.connection.HTTPConnection object at 0x802246da0>, 'Connection to ftp11.tw.freebsd.org timed out. (connect timeout=15)')): False
ALERT:Mirror successful(ftp://ftp15.tw.FreeBSD.org/pub/FreeBSD/), timed out: False
ALERT:Mirror successful(ftp://ftp.ua.FreeBSD.org/pub/FreeBSD/), timed out: False
ALERT:Mirror successful(http://ftp.ua.FreeBSD.org/pub/FreeBSD/), HTTPConnectionPool(host='ftp.ua.freebsd.org', port=80): Max retries exceeded with url: /pub/FreeBSD/releases/ISO-IMAGES/12.0/FreeBSD-12.0-RELEASE-amd64-dvd1.iso (Caused by ConnectTimeoutError(<urllib3.connection.HTTPConnection object at 0x801273240>, 'Connection to ftp.ua.freebsd.org timed out. (connect timeout=15)')): False
ALERT:Mirror successful(ftp://ftp7.ua.FreeBSD.org/pub/FreeBSD/), timed out: False
ALERT:Mirror successful(ftp://ftp2.us.FreeBSD.org/pub/FreeBSD/), timed out: False
ALERT:Mirror successful(ftp://ftp5.us.FreeBSD.org/pub/FreeBSD/), 550 releases/ISO-IMAGES/12.0/: No such file or directory.: False
ALERT:Mirror successful(ftp://ftp10.us.FreeBSD.org/pub/FreeBSD/), 550 Failed to change directory.: False
ALERT:Mirror successful(ftp://ftp13.us.FreeBSD.org/pub/FreeBSD/), timed out: False
ALERT:Mirror successful(http://ftp13.us.FreeBSD.org/pub/FreeBSD/), HTTPConnectionPool(host='ftp13.us.freebsd.org', port=80): Max retries exceeded with url: /pub/FreeBSD/releases/ISO-IMAGES/12.0/FreeBSD-12.0-RELEASE-amd64-dvd1.iso (Caused by ConnectTimeoutError(<urllib3.connection.HTTPConnection object at 0x8022e6940>, 'Connection to ftp13.us.freebsd.org timed out. (connect timeout=15)')): False
ALERT:Mirror successful(ftp://ftp14.us.FreeBSD.org/pub/FreeBSD/), 550 Failed to change directory.: False
ALERT:Mirror successful(http://ftp14.us.FreeBSD.org/pub/FreeBSD/), : False
ALERT:Mirror successful(ftp://ftp15.us.FreeBSD.org/pub/FreeBSD/), 550 Failed to change directory.: False
Comment 1 Gautam Mani 2019-07-10 17:03:42 UTC 
Created attachment 205665 [details]
Mirrors to be checked
Comment 2 Gautam Mani 2019-07-10 17:04:16 UTC 
Created attachment 205666 [details]
Program used to check that mirrors are working.
Comment 3 Gautam Mani 2019-07-10 17:04:54 UTC 
Created attachment 205667 [details]
mirrors.csv
Comment 4 Philip Paeps freebsd_committer freebsd_triage 2019-07-10 17:33:45 UTC 
Thank you very much for doing this work!

I think we should take the out of date mirrors out of the DNS and contact their operators.  We should run your script regularly and add mirrors back to the DNS when they're up to date again.

Li-Wen: can you pull the outdated mirrors from the dns?
Comment 5 Philip Paeps freebsd_committer freebsd_triage 2023-10-06 01:52:26 UTC 
I have contacted several of the mirror operators.  Some stale mirrors are no longer stale and others have been removed.  There is still work to be done here though.  Assign to mirror-admin@ so any clusteradm who has a bit of time can keep following this up.
Comment 6 Philip Paeps freebsd_committer freebsd_triage 2024-02-15 10:35:18 UTC 
I took another pass over the freebsd.org zone and removed several mirrors that were stale or unreachable.

While regional community mirrors were invaluable in the 1990s and early 2000s, they're starting to become a liability in the 2020s.

I've several stale/unreachable mirrors to ftp.geo in the DNS today.  It doesn't cost us anything to keep their names around indefinitely, but at some point in the indeterminate future, we may want to cull them to make the zone less unwieldy.