Bug 254484 - Repication of files put into freefall.freebsd.org/public_distfiles takes a very long time
Summary: Repication of files put into freefall.freebsd.org/public_distfiles takes a ve...
Status: Closed Works As Intended
Alias: None
Product: Services
Classification: Unclassified
Component: Core Infrastructure (show other bugs)
Version: unspecified
Hardware: Any Any
: --- Affects Only Me
Assignee: Philip Paeps
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2021-03-22 19:22 UTC by Yuri Victorovich
Modified: 2021-08-31 06:26 UTC (History)
2 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Yuri Victorovich freebsd_committer freebsd_triage 2021-03-22 19:22:37 UTC
In order to have a custom distfile for a port port maintainers put it into public_distfiles on freefall.freebsd.org.

But then it takes up to an hour or longer for this file to be available in http://distcache.FreeBSD.org/local-distfiles/{user}/, http://distcache.us-east.FreeBSD.org/local-distfiles/{user}/, http://distcache.us-west.FreeBSD.org/local-distfiles/{user}/, etc.

Shouldn't they be replicated on-file-change?

Currently this process can affect some users which wait for a hot-fix, and also delays package building of newly changed ports.
Comment 1 Philip Paeps freebsd_committer freebsd_triage 2021-03-23 01:10:40 UTC
An hour is not a very long time...

I just checked the mechanics: pkg-master fetches local-distfiles from freefall once an hour.  The mirrors sync from pkg-master four times an hour.

I'll change pkg-master to fetch four times an hour too.
Comment 2 Philip Paeps freebsd_committer freebsd_triage 2021-03-24 03:44:39 UTC
pkg-master now pulls from freefall four times an hour (xx:07, xx:22, xx:37 and xx:52).
Comment 3 Yuri Victorovich freebsd_committer freebsd_triage 2021-03-24 03:56:08 UTC
(In reply to Philip Paeps from comment #2)

Thank you!
Comment 4 Yuri Victorovich freebsd_committer freebsd_triage 2021-05-02 17:40:54 UTC
The schedule xx:07, xx:22, xx:37 and xx:52 isn't accurate because when I've put files at 10:11AM PST, and now at 10:40AM PST files are still not replicated.

The problem still exists: 30+ minutes is too long.
Comment 5 Philip Paeps freebsd_committer freebsd_triage 2021-05-07 08:01:53 UTC
As far as I can tell, replication is taking place on the configured schedule.  Which mirror were you checking?

Note that it is not possible to absolutely guarantee that files will replicate everywhere in under 30 minutes.  Several factors affect replication.
Comment 6 Philip Paeps freebsd_committer freebsd_triage 2021-08-31 06:26:51 UTC
As far as I can tell, this is working as intended.  I see the syncs happening four times an hour anyway.  As mentioned, there really is no way to make this "instant" without fundamentally changing how pkgsync works.