Bug 249359 - ports-mgmt/pkg 12.1amd64 repo not updated after at least two portbuilder runs
Summary: ports-mgmt/pkg 12.1amd64 repo not updated after at least two portbuilder runs
Status: Closed FIXED
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Package Infrastructure (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Only Me
Assignee: Li-Wen Hsu
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2020-09-16 04:10 UTC by rkoberman
Modified: 2020-12-13 00:40 UTC (History)
5 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description rkoberman 2020-09-16 04:10:28 UTC
Although pkg-status has shown at least two completed package build runs since "pkg upgrade" last showed any updates on nay of my systems. pkg-status shows the last 12.1amd64 default completed Sun, 13 Sep 2020 01:06:40 GMT.  My systems last updated on Sep. 4. It is likely coincidental, but 1.15 was committed Sep.7.
Comment 1 Antoine Brodin freebsd_committer freebsd_triage 2020-09-16 06:06:07 UTC
Could you check if twn mirror is ok?

freebsd:12:x86:64/latest.1600113437 newer than oldts=1599111617
checking if mirrors have it
mirror rsync://pkg0.isc.freebsd.org/pkg-mirror DOES
mirror rsync://pkg0.nyi.freebsd.org/pkg-mirror DOES
mirror rsync://pkg0.bme.freebsd.org/pkg-mirror DOES
mirror rsync://pkg0.twn.freebsd.org/pkg-mirror does not
mirror rsync://pkg0.tuk.freebsd.org/pkg-mirror DOES
mirror rsync://pkg0.jinx.freebsd.org/pkg-mirror DOES
mirror rsync://pkg0.pkt.freebsd.org/pkg-mirror DOES
mirror state = false
freebsd:12:x86:64/quarterly.1599837830 newer than oldts=1598935593
checking if mirrors have it
mirror rsync://pkg0.isc.freebsd.org/pkg-mirror DOES
mirror rsync://pkg0.nyi.freebsd.org/pkg-mirror DOES
mirror rsync://pkg0.bme.freebsd.org/pkg-mirror DOES
mirror rsync://pkg0.twn.freebsd.org/pkg-mirror does not
mirror rsync://pkg0.tuk.freebsd.org/pkg-mirror DOES
mirror rsync://pkg0.jinx.freebsd.org/pkg-mirror DOES
mirror rsync://pkg0.pkt.freebsd.org/pkg-mirror DOES
mirror state = false
Comment 2 Li-Wen Hsu freebsd_committer freebsd_triage 2020-09-16 07:43:32 UTC
(In reply to Antoine Brodin from comment #1)
Thanks for reporting. Rsync process was stuck, now restarted and now catching up.
Comment 3 mfv 2020-09-16 15:46:10 UTC
Hello,

I had the same experience with amd64 12.1-RELEASE-p9 using pkg v1.14.7.  Since the rsync process hangs up periodically is there a user program that could be used to monitor this process rather than waiting many days before taking appropriate action.

Thanks
Comment 4 rkoberman 2020-09-17 14:49:10 UTC
No idea how long "catching up" should take, ut almost 30 hours and no change in availability of updated repo.
Comment 5 mfv 2020-09-17 16:40:01 UTC
Hello,

Not trying to be pendantic but the last build for default amd64 r12.1 was build number 54692 which ended just after 03:00 UTC.  Thus the last build ended just over 13.5 hours ago. Since this is the 4th build after the rsync process stopped working there is a very large backlog to transmit, especially since one of the builds was over 32.5k packages in size.

But who knows how long it will take to update all the mirrors. The rsync process seem to be quite opaque, as opposed to a very transparent build process.  It is like waiting for a baby.  You never know when the finished product will be delivered.

In this regard here are two suggestions:

 - That 'pkg stats' specify a uniqe identifier, such as a build number or date,
   to identify the local repository; and

 - That there be some way to monitor the progress of the rsync process.

With thanks
Comment 6 Li-Wen Hsu freebsd_committer freebsd_triage 2020-09-17 17:59:58 UTC
(In reply to mfv from comment #5)
Thanks for the suggestions and we will see if it is possible to implement.  There was an issue in the newer version of rsync which prevents it can be restarted after being interrupted. And because all the architectures and branches pkg sets are out of date, it need some (more) time to catch up.  We're temporarily removing this mirror out of the pool and will add it again after it catches up.
Comment 7 rkoberman 2020-09-26 17:31:02 UTC
Looks like it might still be happening. It's been over 2 days since the last build completed with "pkg upgrade" still reporting "up to date" for 12.1amd64 latest.
Comment 8 Philip Paeps freebsd_committer freebsd_triage 2020-12-13 00:40:50 UTC
This was fixed.  But then it happened again a couple of days ago.  All the mirrors are in sync again now.

I like the suggestion in comment #5 though.  I'll see what I can do about adding a way to monitor the mirror sync status.  I don't think it's feasible to show the progress of an individual pkg-master->pkg-mirror sync.  We should however be able to show whether a mirror has the latest sets.

I'll put that somewhere on my (endless) todo list.