Filing a bug so this is not forgotten. I noticed this from daily mail: fetch: https://www.ietf.org/timezones/data/leap-seconds.list: Not Found So there are actually two issues: 1) IEFT is no longer hosting leap-seconds list. One potential source is probably https://hpiers.obspm.fr/iers/bul/bulc/ntp/leap-seconds.list . 2) /etc/periodic/daily/480.leapfile-ntpd should not run "service ntpd onefetch", or "service ntpd oneneedfetch" should not be run. In fact, it's probably reasonable to not run it if ntpd is not enabled.
assign to me for now (in case I can find some time to create a fix), and cc maintainer as backup.
Probably the same as bug 275415.
*** Bug 275415 has been marked as a duplicate of this bug. ***
You might want to combine this one with the following about pretty much same issue https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=275415
Proposed fix at https://reviews.freebsd.org/D42875
For the hosting part, maybe we should host a copy a'd make our scripts fetch from freebsd.org
(In reply to Baptiste Daroussin from comment #6) Agreed.
https://kb.meinbergglobal.com/kb/time_sync/ntp/configuration/ntp_leap_second_file#the_leap_second_file_from_iers has a list of sites. IERS or NIST are the best ones...
To begin with, does the earth's rotation deviate by 1 second that often? :) As I wrote in 275415, our source already has 2 of this file. And the time zone data is updated by freebsd-update and so on. This seems to imply that the minimum update frequency will be met if the file contained in the time zone data is installed...
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=3b3195f6767b39eb33b3523134ef988931c9c86d commit 3b3195f6767b39eb33b3523134ef988931c9c86d Author: Xin LI <delphij@FreeBSD.org> AuthorDate: 2023-12-03 07:00:32 +0000 Commit: Xin LI <delphij@FreeBSD.org> CommitDate: 2023-12-03 07:00:32 +0000 periodic/daily/480.leapfile-ntpd: only attempt to refresh leap-seconds.list when ntpd is enabled. The leap-seconds.list is used exclusively by ntpd, therefore, do not bother to perform the fetch when ntpd is not enabled. PR: conf/275419 Reviewed by: cy, michaelo, imp MFC after: 3 days Differential Revision: https://reviews.freebsd.org/D42875 usr.sbin/periodic/etc/daily/480.leapfile-ntpd | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
*** Bug 275509 has been marked as a duplicate of this bug. ***
A commit in branch stable/14 references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=52369c5d29f5f291bfc98270cf13768633abe322 commit 52369c5d29f5f291bfc98270cf13768633abe322 Author: Xin LI <delphij@FreeBSD.org> AuthorDate: 2023-12-03 07:00:32 +0000 Commit: Xin LI <delphij@FreeBSD.org> CommitDate: 2023-12-06 06:33:48 +0000 periodic/daily/480.leapfile-ntpd: only attempt to refresh leap-seconds.list when ntpd is enabled. The leap-seconds.list is used exclusively by ntpd, therefore, do not bother to perform the fetch when ntpd is not enabled. PR: conf/275419 Reviewed by: cy, michaelo, imp Differential Revision: https://reviews.freebsd.org/D42875 (cherry picked from commit 3b3195f6767b39eb33b3523134ef988931c9c86d) usr.sbin/periodic/etc/daily/480.leapfile-ntpd | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
A commit in branch stable/13 references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=3ef596c6e80562710da09c16558d7351749ea143 commit 3ef596c6e80562710da09c16558d7351749ea143 Author: Xin LI <delphij@FreeBSD.org> AuthorDate: 2023-12-03 07:00:32 +0000 Commit: Xin LI <delphij@FreeBSD.org> CommitDate: 2023-12-06 06:34:26 +0000 periodic/daily/480.leapfile-ntpd: only attempt to refresh leap-seconds.list when ntpd is enabled. The leap-seconds.list is used exclusively by ntpd, therefore, do not bother to perform the fetch when ntpd is not enabled. PR: conf/275419 Reviewed by: cy, michaelo, imp Differential Revision: https://reviews.freebsd.org/D42875 (cherry picked from commit 3b3195f6767b39eb33b3523134ef988931c9c86d) usr.sbin/periodic/etc/daily/480.leapfile-ntpd | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
A commit in branch stable/12 references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=80560eec655ac6cd3b7d7fd0f9fe37f309b78d64 commit 80560eec655ac6cd3b7d7fd0f9fe37f309b78d64 Author: Xin LI <delphij@FreeBSD.org> AuthorDate: 2023-12-03 07:00:32 +0000 Commit: Xin LI <delphij@FreeBSD.org> CommitDate: 2023-12-06 07:55:17 +0000 periodic/daily/480.leapfile-ntpd: only attempt to refresh leap-seconds.list when ntpd is enabled. The leap-seconds.list is used exclusively by ntpd, therefore, do not bother to perform the fetch when ntpd is not enabled. PR: conf/275419 Reviewed by: cy, michaelo, imp Differential Revision: https://reviews.freebsd.org/D42875 (cherry picked from commit 3b3195f6767b39eb33b3523134ef988931c9c86d) usr.sbin/periodic/etc/daily/480.leapfile-ntpd | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
Should we consider this for a 14.0 erratum update?
(In reply to Ed Maste from comment #15) Yes (along with leap-seconds update (2acec38b91fdf8520ca1ca6aebbde5c527040d14 / 832c63f5e07e6f960d051747ccb4358695a393ec)). I'm still waiting for a decision about where do we fetch the leap seconds file from, ideally we should do all three changes together, but this + leap second update should buy us some time too.
Either nist or bipm will be authoritative. I'm inclined to fetch from a list that includes these two. Nist has been there forever and bimp is likely going to be long term stable. Nrao was retired several years ago. It's the easy button... the idea of setting up a FreeBSD server for that seems like overkill. Not sure why we did ietf at all... but if we do a list, we are quite safe. If we really want to stand up a FreeBSD server we can add it to the list in a future release. Just my two cents...
I have not yet updated comment #14 , so I have made the following settings in /etc/rc.conf. ntp_leapfile_sources="file:///usr/src/contrib/tzdata/leap-seconds.list" Perhaps this is the most powerful :) quote from comment #17 > Not sure why we did ietf at all... The distribution source of this contrib/tzdata is presumed to be ietf. Recently, the patch version has been pushed up even with the update of tzdata. https://www.freebsd.org/security/advisories/FreeBSD-EN-23:05.tzdata.asc ... so, do we need fetch?
(In reply to Tatsuki Makino from comment #18) Well, there are some users who don't install src/ at all. Our rc.d script actually does check /etc/ntp/leap-seconds prior to perform the fetch. An EN with the updated leap seconds should be sufficient to prevent fetching. I have some concerns with the IERS version as the licensing terms is not really clear (I haven't done deeper research so I could be wrong here). NIST copy is in public domain (because it's US Government) and is therefore preferred in the past, but it's only served on FTP. We should probably just host it at www.freebsd.org somewhere...
Bipm is also free. It's identical to nist's file... But philipbhas committed a different fix.
(In reply to Ed Maste from comment #15) Yes please. I have had good success with this 'patch' so far: $ sudo sysrc ntp_leapfile_sources="ftp://ftp.boulder.nist.gov/pub/time/leap-seconds.list"
It is not considered nice to hard code something where we have no control. In the ntp community, the story of one university's stratum 1 ntp service is probably well known to those in the know :) Once again I am re-writing here, we already have two files. [1] file:///usr/src/usr.sbin/ntp/ntpd/leap-seconds [2] file:///usr/src/contrib/tzdata/leap-seconds.list [1] is the source of the files installed in /etc/ntp, recently updated by Xin-san. If the log of file [1] is traced back. When the logs are traced back by git, https://cgit.freebsd.org/src/log/etc/ntp/leap-seconds should also be looked at. It starts with a log that looks like it was added for use with ntpd. https://cgit.freebsd.org/src/commit/etc/ntp/leap-seconds?id=6f422073d1099f91b6af5cd586354f8662111819 If the log of file [2] is traced back. We are forced to move from one directory to another, but this file seems to have a longer history. This is the commit where it was made to be included in the source. https://cgit.freebsd.org/src/commit/contrib/tzdata/leap-seconds.list?id=c26a580251c8b7c4ea43a8204188bb4ff0193455 This seems to be the first commit in which this file appeared. https://cgit.freebsd.org/src/commit/leap-seconds.list?h=vendor/tzdata&id=59b8fbef6c807537f08ce49bdf9db73f7f9e07f6 Having written this far, I am not sure what I was trying to say :) First, the [1] and [2] files should be integrated in their usage. I join the faction that says [2] is the better file to be installed by default :) It is expected to be new with the frequency with which time zone data is updated. If updates are made by fetch, this frequency of updates should be sufficient by default. As an addition, these files don't exist in www, but we are in a position to say that they are hosted in cgit, aren't we? However, it would have to be copied to www or www would have to be a proxy since it would not be a place than for such a use.
The difference is that (a) these files are updated 2x a year at most for at least the next 20 years and (b) these urls are published by nist and bipm as being open to the public. Taken together, we don't need to engineer a high volume robust solution. The existing infrastructure is sufficient and it becomes insufficient it will be with a lot of warning. Plus, the file is a backup to misbehaving reference clocks... so it's need for this use isn't super urgent. Finally, there is a big push to stop leaps entirely so the ROI on a complex infrastructure is likely to be low.
In case this helps: I've set: [0:15 r730-01 dvl ~] % grep ntp /etc/rc.conf ntpd_flags="-L" daily_ntpd_leapfile_enable="YES" daily_ntpd_avoid_congestion="YES" ntpd_enable="YES" ntp_leapfile_sources="ftp://ftp.boulder.nist.gov/pub/time/leap-seconds.list" ntp_leapfile_fetch_verbose="YES" When fetching: [0:15 r730-01 dvl ~] % sudo service ntpd fetch ntp_src_leapfile version is 3676924800 expires 3912710400 ntp_db_leapfile version is 3676924800 expires 3912710400 not replacing /var/db/ntpd.leap-seconds.list with /etc/ntp/leap-seconds Within ntp leapfile expiry limit, initiating fetch fetching ftp://ftp.boulder.nist.gov/pub/time/leap-seconds.list using existing /var/db/ntpd.leap-seconds.list The file referenced above: [0:16 r730-01 dvl ~] % ls -l /var/db/ntpd.leap-seconds.list -rw-r--r-- 1 root wheel 10666 2023.03.28 20:25 /var/db/ntpd.leap-seconds.list When fetched manually, I get: -rw-r--r-- 1 dvl dvl 10917 2023.08.06 15:25 leap-seconds.list
(In reply to Dan Langille from comment #24) Interesting that ftp.boulder.nist.gov has A and AAAA records but only seems to be listening on the IPv4 address (and does not respond to ping).
Leap seconds will be eliminated... (ITU WRC Dubai 2023) 🤣
Draft for the errata. ============================================================================= FreeBSD-EN-23:XX.ntp Errata Notice The FreeBSD Project Topic: NTP Category: contrib Module: ntp Announced: XXXX-XX-XX Affects: All supported FreeBSD releases Corrected: XXXX-XX-XX XX:XX:XX UTC (stable/14, 14.0-STABLE) XXXX-XX-XX XX:XX:XX UTC (releng/14.0, 14.0-RELEASE-pXX) XXXX-XX-XX XX:XX:XX UTC (stable/13, 13.2-STABLE) XXXX-XX-XX XX:XX:XX UTC (releng/13.2, 13.2-RELEASE-pXX) For general information regarding FreeBSD Errata Notices and Security Advisories, including descriptions of the fields above, security branches, and the following sections, please visit <URL:https://security.FreeBSD.org/>. I. Background The Network Time Protocol (NTP) daemon (ntpd) synchronizes system time with timeservers over a network. This helps ensure accurate timekeeping, which is crucial for various applications, including network protocols, logging, and security. Leap seconds are occasional adjustments to Coordinated Universal Time (UTC) to account for the Earth's slightly irregular rotation. These adjustments are infrequent and announced in advance. The ntpd daemon relies on a leap second list file to handle these adjustments correctly. FreeBSD uses a leap second list file (/etc/ntp/leap-seconds) that contains the schedule of future leap seconds. This file has an expiration date. When the file expires, the system automatically attempts to download a new list from a pre-configured source. II. Problem Description Expired leap seconds list: The current leap second list file on FreeBSD systems will expire on December 31, 2023. As the result, systems will attempt to download a new list from the previously configured source, which is the International Engineering Task Force (IETF) server. Obsolete download Location: Unfortunately, the IETF server no longer serves the leap seconds list file. This means that affected systems will be unable to download a new list when needed. Unnecessary downloads: Currently, all FreeBSD systems attempt to download the leap seconds list file, regardless of whether they are running the ntpd daemon. For systems that are not running the ntp daemon, this is not an efficient use of resources. III. Impact The daily periodic script would attempt to download and complain about being unable to download the updated leap seconds list. IV. Workaround Set daily_ntpd_leapfile_enable to "NO" in /etc/periodic.conf . Note that this means the leap second list will not be updated. V. Solution Upgrade your system to a supported FreeBSD stable or release / security branch (releng) dated after the correction date. A reboot is required after the upgrade procedure has been completed. Perform one of the following: 1) To update your system via a binary patch: Systems running a RELEASE version of FreeBSD on the amd64 or arm64 platforms, or the i386 platfrom on FreeBSD 13, can be updated via the freebsd-update(8) utility: # freebsd-update fetch # freebsd-update install # shutdown -r now 2) To update your system via a source code patch: The following patches have been verified to apply to the applicable FreeBSD release branches. a) Download the relevant patch from the location below, and verify the detached PGP signature using your PGP utility. # fetch https://security.FreeBSD.org/patches/EN-23:XX/ntp.patch # fetch https://security.FreeBSD.org/patches/EN-23:XX/ntp.patch.asc # gpg --verify ntp.patch.asc b) Apply the patch. Execute the following commands as root: # cd /usr/src # patch < /path/to/patch c) Recompile the operating system using buildworld and installworld as described in <URL:https://www.FreeBSD.org/handbook/makeworld.html>. VI. Correction details This issue is corrected as of the corresponding Git commit hash or Subversion revision number in the following stable and release branches: Branch/path Hash Revision - ------------------------------------------------------------------------- stable/14/ XXXXXXXXXXXX stable/14-nXXXXXX releng/14.0/ XXXXXXXXXXXX releng/14.0-nXXXXXX stable/13/ XXXXXXXXXXXX stable/13-nXXXXXX releng/13.2/ XXXXXXXXXXXX releng/13.2-nXXXXXX - ------------------------------------------------------------------------- Run the following command to see which files were modified by a particular commit: # git show --stat <commit hash> Or visit the following URL, replacing NNNNNN with the hash: <URL:https://cgit.freebsd.org/src/commit/?id=NNNNNN> To determine the commit count in a working tree (for comparison against nNNNNNN in the table above), run: # git rev-list --count --first-parent HEAD VII. References <URL:https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=275419> The latest revision of this advisory is available at <URL:https://security.FreeBSD.org/advisories/FreeBSD-EN-23:XX.ntp.asc>
Send to secteam@'s queue for EN
As pointed out, I've changed main to try to fetch leap-seconds.list from IANA instead of IETF. The IERS is authoritative-authoritative, but there are copyright concerns (no public domain in France). NIST's version would be better but it's only served over FTP and it's 2023. There is no rush for everyone to update their leap-seconds.list files. There will be no leap second at the end of 2023 so the installed version is correct. All updating will do is silence an ntpd diagnostic. It may be worth doing an EN to stop systems not running NTP from trying to download the file.
*** Bug 275737 has been marked as a duplicate of this bug. ***
*** Bug 275799 has been marked as a duplicate of this bug. ***
I have tried various suggested values for ntp_leapfile_sources in /etc/rc.conf and for each I have run: run service ntpd fetch - for example: [19:27 r730-01 dvl ~] % sudo service ntpd fetch ntp_src_leapfile version is 3676924800 expires 3912710400 ntp_db_leapfile version is 3676924800 expires 3912710400 not replacing /var/db/ntpd.leap-seconds.list with /etc/ntp/leap-seconds Within ntp leapfile expiry limit, initiating fetch fetching ftp://ftp.boulder.nist.gov/pub/time/leap-seconds.list using existing /var/db/ntpd.leap-seconds.list The original problem persists and this is logged when restarting ntpd: Dec 19 19:31:36 r730-01 ntpd[66134]: leapsecond file ('/var/db/ntpd.leap-seconds.list'): will expire in less than 9 days Am I doing this wrong?
(In reply to Dan Langille from comment #32) No, the file on https://data.iana.org/time-zones/tzdb/leap-seconds.list expires on 2023-12-08, indeed: # curl -s https://data.iana.org/time-zones/tzdb/leap-seconds.list | grep "expires on" # File expires on: 28 December 2023 The file at IERS is apparently newer, as it expires on 2024-06-28: # curl -s https://hpiers.obspm.fr/iers/bul/bulc/ntp/leap-seconds.list | grep "expires on" # File expires on 28 June 2024 Somebody should tell IETF to update their copy. :)
(In reply to Dimitry Andric from comment #33) Success: [14:02 r730-01 dvl ~] % sudo sysrc ntp_leapfile_sources=https://hpiers.obspm.fr/iers/bul/bulc/ntp/leap-seconds.list ntp_leapfile_sources: https://cgit.freebsd.org/src/plain/contrib/tzdata/leap-seconds.list -> https://hpiers.obspm.fr/iers/bul/bulc/ntp/leap-seconds.list [14:03 r730-01 dvl ~] % sudo service ntpd fetch ntp_src_leapfile version is 3676924800 expires 3912710400 ntp_db_leapfile version is 3676924800 expires 3912710400 not replacing /var/db/ntpd.leap-seconds.list with /etc/ntp/leap-seconds Within ntp leapfile expiry limit, initiating fetch fetching https://hpiers.obspm.fr/iers/bul/bulc/ntp/leap-seconds.list using https://hpiers.obspm.fr/iers/bul/bulc/ntp/leap-seconds.list as /var/db/ntpd.leap-seconds.list [14:03 r730-01 dvl ~] % sudo service ntpd restart Stopping ntpd. Waiting for PIDS: 66134. Starting ntpd. [14:03 r730-01 dvl ~] % tail /var/log/messages Dec 20 14:03:21 r730-01 ntpd[1734]: ntpd 4.2.8p16-a (1): Starting Dec 20 14:03:21 r730-01 ntpd[1734]: Command line: /usr/sbin/ntpd -L -p /var/db/ntp/ntpd.pid -c /etc/ntp.conf -f /var/db/ntp/ntpd.drift Dec 20 14:03:21 r730-01 ntpd[1734]: ---------------------------------------------------- Dec 20 14:03:21 r730-01 ntpd[1734]: ntp-4 is maintained by Network Time Foundation, Dec 20 14:03:21 r730-01 ntpd[1734]: Inc. (NTF), a non-profit 501(c)(3) public-benefit Dec 20 14:03:21 r730-01 ntpd[1734]: corporation. Support and training for ntp-4 are Dec 20 14:03:21 r730-01 ntpd[1734]: available at https://www.nwtime.org/support Dec 20 14:03:21 r730-01 ntpd[1734]: ---------------------------------------------------- Dec 20 14:03:21 r730-01 ntpd[1739]: leapsecond file ('/var/db/ntpd.leap-seconds.list'): good hash signature Dec 20 14:03:21 r730-01 ntpd[1739]: leapsecond file ('/var/db/ntpd.leap-seconds.list'): loaded, expire=2024-06-28T00:00:00Z last=2017-01-01T00:00:00Z ofs=37 [14:03 r730-01 dvl ~] %
Since there's no issue with grabbing from BIPM, let's just do that. this alternative source is too late in updating to the new file my a few months. This issue is why I wanted to go to source of truth every time for all our users and not get any other intermediaries in the way. If our users don't trust this source, they can change it to some place else. The 'trust' issue is undermined if we funnel them to an unreliable source of data. Permission for the bipm link can be found in https://hpiers.obspm.fr/iers/bul/bulc/README which states # The files in this directory and sub directories are licensed under the Creative Commons Attribution 4.0 : CC BY-ND 4.0 # http://creativecommons.org/licenses/by/4.0/ If we distribute it, then we need to acknowledge where it came from. And no derivatives are permitted. We are making no changes to this file, so no derivatives. Our users are downloading it, which is totally fine by CC BY-ND since they aren't redistributing it. Since we're not redistributing it ourselves, no further action is required by us. Though perhaps we need the above link in the .defaults file that lists this URL. IMHO, It would be foolish of us to issue an advisory pointing to a source that has a file that expires in a week (even if they fix it in the mean time).
(In reply to Dimitry Andric from comment #33) FYI, since making that change (and posting it twice), I have not seen those log messages again. Thank you.
(In reply to Dimitry Andric from comment #33) I have contacted the IETF and they responded that the canonical file is indeed https://data.iana.org/time-zones/data/leap-seconds.list and they won't be updating the ietf site to serve this file anymore. The IANA file has been updated to expire on 28 June 2024.
Since IANA have updated their expiration date, we're almost good to close this bug as "overcome by events". However, I will leave this as "in progress" until we get an errata notice and freebsd-update builds out with tzdata2023d and the IANA URL in defaults instead of the IETF.
A commit in branch releng/14.0 references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=a3b7bafd2acc4ddfba18c8ad990b29c407892ed6 commit a3b7bafd2acc4ddfba18c8ad990b29c407892ed6 Author: Xin LI <delphij@FreeBSD.org> AuthorDate: 2023-12-03 07:00:32 +0000 Commit: Philip Paeps <philip@FreeBSD.org> CommitDate: 2024-02-14 06:19:01 +0000 periodic/daily/480.leapfile-ntpd: only attempt to refresh leap-seconds.list when ntpd is enabled. The leap-seconds.list is used exclusively by ntpd, therefore, do not bother to perform the fetch when ntpd is not enabled. PR: conf/275419 Reviewed by: cy, michaelo, imp Differential Revision: https://reviews.freebsd.org/D42875 (cherry picked from commit 3b3195f6767b39eb33b3523134ef988931c9c86d) (cherry picked from commit 52369c5d29f5f291bfc98270cf13768633abe322) Security: FreeBSD-EN-24:01.tzdata Approved by: so (gordon) usr.sbin/periodic/etc/daily/480.leapfile-ntpd | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
A commit in branch releng/13.2 references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=66bb668fe5f2561844f5b79251ea42e1bfce9aee commit 66bb668fe5f2561844f5b79251ea42e1bfce9aee Author: Xin LI <delphij@FreeBSD.org> AuthorDate: 2023-12-03 07:00:32 +0000 Commit: Philip Paeps <philip@FreeBSD.org> CommitDate: 2024-02-14 06:25:34 +0000 periodic/daily/480.leapfile-ntpd: only attempt to refresh leap-seconds.list when ntpd is enabled. The leap-seconds.list is used exclusively by ntpd, therefore, do not bother to perform the fetch when ntpd is not enabled. PR: conf/275419 Reviewed by: cy, michaelo, imp Differential Revision: https://reviews.freebsd.org/D42875 (cherry picked from commit 3b3195f6767b39eb33b3523134ef988931c9c86d) (cherry picked from commit 3ef596c6e80562710da09c16558d7351749ea143) Security: FreeBSD-EN-24:01.tzdata Approved by: so (gordon) usr.sbin/periodic/etc/daily/480.leapfile-ntpd | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
FreeBSD-EN-24:01.tzdata updates our contrib/tzdata to tzdata 2024a, and also includes the two fixes from this PR: fetch leap-seconds.list from IERS (Warner) and teach periodic not to run unnecessarily (Xin).