Bug 209577

Summary: rc.d/ntpd relies on inconsistent file timestamps
Product: Base System Reporter: rwmaillists
Component: miscAssignee: Cy Schubert <cy>
Status: Closed FIXED    
Severity: Affects Some People CC: cy, des, mvharding
Priority: --- Flags: koobs: mfc-stable10+
koobs: mfc-stable9+
Version: 10.3-RELEASE   
Hardware: Any   
OS: Any   
Attachments:
Description Flags
Proposed patch to use expiry rather than verison number. none

Description rwmaillists 2016-05-17 11:53:28 UTC
It seems that different leap-second file sources use different time-stamps. The IETF doesn't  change the time-stamp when the only the expiry changes, but the IERS does. 

This wont affect the leap-seconds seen by ntpd, but it can prevent rc.d/ntpd from replacing an expired /var/db/ntpd.leap-seconds.list file causing a warning and fetch(1) trying to mirror the file every day.
Comment 1 Cy Schubert freebsd_committer freebsd_triage 2016-05-20 14:14:28 UTC
The issue is that the ietf leapfile is fetchable
Comment 2 Cy Schubert freebsd_committer freebsd_triage 2016-05-20 14:17:26 UTC
The IETF source is https whereas the other sources are FTP. The rc.d/ntpd avoids ftp as some sites may restrict due to firewall issues. You can add ntp_leapfile_sources= to /etc/rc.conf to alter the source.
Comment 3 rwmaillists 2016-05-20 17:13:09 UTC
It's not really a question of where they are downloaded from, but how they are compared.

The issue is that it's possible to download a more up-to-date copy with a later expiry, but with the same (or even an earlier) timestamp. rc.d/ntpd will only overwrite the working copy if the downloaded copy has a later timestamp, not a later expiry.

As things stand a file installed from /etc/ntp/ to /var/db/ wont be overwritten by the IETF file until a new leap-second is announced. Five months after that, we're back where we are now.
Comment 4 Cy Schubert freebsd_committer freebsd_triage 2016-05-20 19:53:16 UTC
The three locations the file can be fetched from are ftp://time.nist.gov/pub/, ftp://tycho.usno.navy.mil/pub/ntp/ and  https://www.ietf.org/timezones/data/leap-seconds.list.

Comparing the copy from ftp://tycho.usno.navy.mil/pub/ntp/ with the one from https://www.ietf.org/timezones/data/leap-seconds.list, not much difference exists.

173,174c202,203
< #	Updated through IERS Bulletin C 51
< #	File expires on:  1 Dec 2016
---
> #	Updated through IERS Bulletin C51
> #	File expires on:  28 December 2016
176c205
< #@ 3689539200
---
> #@ 3691872000

I do not know where to fetch the IERS version of the file from. Having said that apparently the IERS version is copyrighted: https://mm.icann.org/pipermail/tz/2016-February/023171.html. I haven't been able to confirm this.
Comment 5 rwmaillists 2016-05-23 10:28:10 UTC
I originally brought this up because someone had a problem with the IERS copy. I probably didn't make myself clear in the previous post, but the script doesn't work properly even if you stay with the IETF version. The downloaded copy isn't overwriting the working copy from the 10.3 install because they have different expiry times but the same time-stamp.

The reason for checking the timestamp is to prevent the expiry going backwards, comparing the actual expiry time would make everything just work.
Comment 6 Cy Schubert freebsd_committer freebsd_triage 2016-05-23 20:49:22 UTC
Created attachment 170580 [details]
Proposed patch to use expiry rather than verison number.

The attached patch uses expiry timestamp rather than version number (creation timestamp) of the file. It's functionally the same but fetches and replaces the file even if the contents, excluding version numbers, is the same.
Comment 7 Cy Schubert freebsd_committer freebsd_triage 2016-05-23 20:55:59 UTC
Rather than use the leapfile serial number (create timestamp) we can use the expiry date instead.
Comment 8 commit-hook freebsd_committer freebsd_triage 2016-05-25 01:35:15 UTC
A commit references this bug:

Author: cy
Date: Wed May 25 01:35:02 UTC 2016
New revision: 300638
URL: https://svnweb.freebsd.org/changeset/base/300638

Log:
  Use the expiry date to determine whether to replace the DB copy of
  leapfile instead of using the leapfile serial number (create
  timestamp).

  PR:		209577
  MFC after:	3 days

Changes:
  head/etc/rc.d/ntpd
Comment 9 commit-hook freebsd_committer freebsd_triage 2016-05-28 03:33:43 UTC
A commit references this bug:

Author: cy
Date: Sat May 28 03:33:06 UTC 2016
New revision: 300897
URL: https://svnweb.freebsd.org/changeset/base/300897

Log:
  MFC r300638:

  Use the expiry date to determine whether to replace the DB copy of
  leapfile instead of using the leapfile serial number (create
  timestamp).

  PR:		209577

Changes:
_U  stable/10/
  stable/10/etc/rc.d/ntpd
Comment 10 commit-hook freebsd_committer freebsd_triage 2016-05-28 03:34:45 UTC
A commit references this bug:

Author: cy
Date: Sat May 28 03:34:00 UTC 2016
New revision: 300898
URL: https://svnweb.freebsd.org/changeset/base/300898

Log:
  MFC r300638:

  Use the expiry date to determine whether to replace the DB copy of
  leapfile instead of using the leapfile serial number (create
  timestamp).

  PR:		209577

Changes:
_U  stable/9/etc/
_U  stable/9/etc/rc.d/
  stable/9/etc/rc.d/ntpd
Comment 11 Cy Schubert freebsd_committer freebsd_triage 2016-06-14 03:55:30 UTC
Fix committed.
Comment 12 Cy Schubert freebsd_committer freebsd_triage 2016-06-14 03:55:46 UTC
*** Bug 209970 has been marked as a duplicate of this bug. ***