Bug 217055 - Consolidate random sleeps in periodic scripts
Summary: Consolidate random sleeps in periodic scripts
Status: Closed FIXED
Alias: None
Product: Base System
Classification: Unclassified
Component: conf (show other bugs)
Version: CURRENT
Hardware: Any Any
: --- Affects Many People
Assignee: Alan Somers
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2017-02-13 05:28 UTC by Alan Somers
Modified: 2017-04-24 16:15 UTC (History)
1 user (show)

See Also:
asomers: mfc-stable11+


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Alan Somers freebsd_committer freebsd_triage 2017-02-13 05:28:03 UTC
On my system, there are three separate periodic scripts that try to sleep for a random amount of time to reduce congestion on a download server somewhere: 300.statistics (from sysutils/bsdstat), 410.pkg-audit (from ports-mgmt/pkg), and 480.leapfile-ntpd (from the base).  There may be more throughout the ports tree.  A single sleep would be sufficient, but if all three scripts decide to sleep, the total sleep time can get quite long.

I propose we add a 000.anticongestion periodic script, enabled by default for daily, weekly, and monthly.  It will sleep for 0 - 60 minutes, unless periodic is being run interactively.  Then we can remove the sleep in 480.leapfile-ntpd, and conditionalize the ports sleeps on __FreeBSD_version.
Comment 1 Alan Somers freebsd_committer freebsd_triage 2017-02-13 05:28:23 UTC
I'll take it.
Comment 2 Brad Davis freebsd_committer freebsd_triage 2017-02-13 16:27:33 UTC
I think this is a good idea.

Especially with a variable so you can override the randomness and gain some control of when your scripts run.  Like a variable to set the delay, i.e.:

anticongestion_sleep_override=900
Comment 3 commit-hook freebsd_committer freebsd_triage 2017-04-01 04:44:45 UTC
A commit references this bug:

Author: asomers
Date: Sat Apr  1 04:42:35 UTC 2017
New revision: 316342
URL: https://svnweb.freebsd.org/changeset/base/316342

Log:
  Consolidate random sleeps in periodic scripts

  Multiple periodic scripts sleep for a random amount of time in order to
  mitigate the thundering herd problem. This is bad, because the sum of
  multiple uniformly distributed random variables approaches a normal
  distribution, so the problem isn't mitigated as effectively as it would be
  with a single sleep.

  This change creates a single configurable anticongestion sleep. periodic
  will only sleep if at least one script requires it, and it will never sleep
  more than once per invocation. It also won't sleep if periodic was run
  interactively, fixing an unrelated longstanding bug.

  PR:		217055
  PR:		210188
  Reviewed by:	cy
  MFC after:	3 weeks
  Differential Revision:	https://reviews.freebsd.org/D10211

Changes:
  head/etc/defaults/periodic.conf
  head/etc/periodic/daily/480.leapfile-ntpd
  head/share/man/man5/periodic.conf.5
  head/usr.sbin/periodic/periodic.sh
Comment 4 commit-hook freebsd_committer freebsd_triage 2017-04-24 15:29:44 UTC
A commit references this bug:

Author: asomers
Date: Mon Apr 24 15:29:18 UTC 2017
New revision: 317373
URL: https://svnweb.freebsd.org/changeset/base/317373

Log:
  MFC r316342, r316358

  r316342:
  Consolidate random sleeps in periodic scripts

  Multiple periodic scripts sleep for a random amount of time in order to
  mitigate the thundering herd problem. This is bad, because the sum of
  multiple uniformly distributed random variables approaches a normal
  distribution, so the problem isn't mitigated as effectively as it would be
  with a single sleep.

  This change creates a single configurable anticongestion sleep. periodic
  will only sleep if at least one script requires it, and it will never sleep
  more than once per invocation. It also won't sleep if periodic was run
  interactively, fixing an unrelated longstanding bug.

  PR:		217055
  PR:		210188
  Reviewed by:	cy
  MFC after:	3 weeks
  Differential Revision:	https://reviews.freebsd.org/D10211

  r316358:
  Fix man page typo from r316342

  Reported by:	rgrimes
  MFC after:	20 days
  X-MFC-With:	316342

Changes:
_U  stable/11/
  stable/11/etc/defaults/periodic.conf
  stable/11/etc/periodic/daily/480.leapfile-ntpd
  stable/11/share/man/man5/periodic.conf.5
  stable/11/usr.sbin/periodic/periodic.sh