Bug 262075

Summary:	database/mariadb*: Improve MASTER_SITES sources and selection
Product:	Ports & Packages	Reporter:	Daniel Engberg <diizzy>
Component:	Individual Port(s)	Assignee:	Bernard Spil <brnrd>
Status:	Closed FIXED
Severity:	Affects Many People	CC:	daniel
Priority:	---	Keywords:	needs-patch, needs-qa
Version:	Latest	Flags:	koobs: merge-quarterly?
Hardware:	Any
OS:	Any

Description Daniel Engberg freebsd_committer

2022-02-20 21:14:06 UTC

Hi,

All ports seems to have "random" mirror sites, some dead, unofficial etc and the newest addition have 3 dead mirrors before fetch finds one working...

=> Attempting to fetch http://mirrors.supportex.net/mariadb/mariadb-10.6.7/source/mariadb-10.6.7.tar.gz
fetch: http://mirrors.supportex.net/mariadb/mariadb-10.6.7/source/mariadb-10.6.7.tar.gz: size mismatch: expected 85882896, actual 0
=> Attempting to fetch http://mirror2.hs-esslingen.de/pub/Mirrors/mariadb/mariadb-10.6.7/source/mariadb-10.6.7.tar.gz
fetch: http://mirror2.hs-esslingen.de/pub/Mirrors/mariadb/mariadb-10.6.7/source/mariadb-10.6.7.tar.gz: Not Found
=> Attempting to fetch http://gd.tuwien.ac.at/db/mariadb/mariadb-10.6.7/source/mariadb-10.6.7.tar.gz
fetch: http://gd.tuwien.ac.at/db/mariadb/mariadb-10.6.7/source/mariadb-10.6.7.tar.gz: No address record
=> Attempting to fetch http://mirrors.fe.up.pt/pub/mariadb/mariadb-10.6.7/source/mariadb-10.6.7.tar.gz
mariadb-10.6.7.tar.gz                                   81 MB  768 kBps 01m49s

Please consider adding a section in https://cgit.freebsd.org/ports/tree/Mk/bsd.sites.mk for MariaDB mirrors

A suggestion would be something like this as these all have good overall connectivity and diversity so if all these fails we can safely assume it's not a connectivity issue. URLs needs to be adapted to requested version.

https://mirror.nodesdirect.com/mariadb/mariadb-10.7.3/source/mariadb-10.7.3.tar.gz
https://mirror.netcologne.de/mariadb/mariadb-10.7.3/source/mariadb-10.7.3.tar.gz
https://ftp.harukasan.org/mariadb/mariadb-10.7.3/source/mariadb-10.7.3.tar.gz
https://mirror.its.dal.ca/mariadb/mariadb-10.7.3/source/mariadb-10.7.3.tar.gz
https://mirror.one.com/mariadb/mariadb-10.7.3/source/mariadb-10.7.3.tar.gz
https://ftp.yz.yamagata-u.ac.jp/pub/dbms/mariadb/mariadb-10.7.3/source/mariadb-10.7.3.tar.gz
https://mirror.one.com/mariadb/mariadb-10.7.3/source/mariadb-10.7.3.tar.gz
https://ftp.osuosl.org/pub/mariadb/mariadb-10.7.3/source/mariadb-10.7.3.tar.gz

Taken from https://mariadb.org/download/?t=mariadb&o=true&p=mariadb&r=10.3.34&os=source&m=osuosl (site list)

Best regards,
Daniel

Comment 1 Daniel Engberg freebsd_committer

2022-04-01 07:13:11 UTC

@Bernard

Friendly ping

Comment 2 Bernard Spil freebsd_committer

2022-04-18 11:02:08 UTC

Hi,

Comment 3 Bernard Spil freebsd_committer

2022-04-18 11:17:35 UTC

Did a bit of searching, came up with 

1. Redirect by mariadb
2. Secondary rsync mirrors
3. Primary rsync mirror

MASTER_SITES=   https://downloads.mariadb.org/rest-api/mariadb/${PORTVERSION}/ \
                https://mirror.nodesdirect.com/${SITESDIR}/ \
                https://mirror.one.com/${SITESDIR}/ \
                https://ftp.nluug.nl/db/${SITESDIR}/ \
                https://ftp.icm.edu.pl/pub/unix/database/${SITESDIR}/ \
                https://mirror.truenetwork.ru/${SITESDIR}/ \
                https://mirrors.aliyun.com/${SITESDIR}/ \
                https://ftp.ubuntu-tw.org/mirror/${SITESDIR}/ \
                https://mirror.kku.ac.th/${SITESDIR}/ \
                https://espejito.fder.edu.uy/${SITESDIR}/ \
                https://mirror1.cl.netactuate.com/${SITESDIR}/ \
                https://ftp.osuosl.org/pub/${SITESDIR}/

Info gathered from
* https://mariadb.com/kb/en/mirror-sites-for-mariadb/
* https://mirmon.mariadb.org/
* https://mariadb.org/downloads-rest-api/
* https://downloads.mariadb.org/rest-api/mirrors

Comment 4 commit-hook freebsd_committer

2022-04-18 11:25:09 UTC

A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/ports/commit/?id=5ab12f1ecf3f97ae049497bce765c17919075e0b

commit 5ab12f1ecf3f97ae049497bce765c17919075e0b
Author:     Bernard Spil <brnrd@FreeBSD.org>
AuthorDate: 2022-04-18 11:22:30 +0000
Commit:     Bernard Spil <brnrd@FreeBSD.org>
CommitDate: 2022-04-18 11:22:30 +0000

    databases/mariadb*: Update MASTER_SITES

     * Fixes broken sites
     * Prefer MariaDB redirect mechanism via API
     * Use secondary mirrors
     * Primary mirror as last resort

    PR:             262075
    Reported by:    diizzy

 databases/mariadb-connector-c/Makefile    | 19 +++++++++++--------
 databases/mariadb-connector-odbc/Makefile | 19 +++++++++++--------
 databases/mariadb103-server/Makefile      | 20 ++++++++++++--------
 databases/mariadb104-server/Makefile      | 20 ++++++++++++--------
 databases/mariadb105-server/Makefile      | 19 ++++++++++++-------
 databases/mariadb106-server/Makefile      | 20 ++++++++++++--------
 6 files changed, 70 insertions(+), 47 deletions(-)

Comment 5 Daniel Engberg freebsd_committer

2022-04-18 11:30:45 UTC

Nice, thanks!

Comment 6 Daniel Black 2022-04-27 04:03:19 UTC

URLs of the form https://mirror.mariadb.org/mariadb-10.6.7/source/mariadb-10.6.7.tar.gz exist too now as a multi-site re-director (https://mirror.mariadb.org/mirrorstats).