Bug 210188
| Summary: | periodic daily sleeps even when invoked from a terminal | ||
|---|---|---|---|
| Product: | Base System | Reporter: | Martin S. Weber <freebsd-bugzilla> |
| Component: | bin | Assignee: | Alan Somers <asomers> |
| Status: | Closed FIXED | ||
| Severity: | Affects Only Me | CC: | asomers |
| Priority: | --- | Flags: | asomers:
mfc-stable11+
|
| Version: | 10.3-RELEASE | ||
| Hardware: | Any | ||
| OS: | Any | ||
Looks like this behavior was added back in 2000 by revision 65843, probably unintentionally. A commit references this bug: Author: asomers Date: Sat Apr 1 04:42:35 UTC 2017 New revision: 316342 URL: https://svnweb.freebsd.org/changeset/base/316342 Log: Consolidate random sleeps in periodic scripts Multiple periodic scripts sleep for a random amount of time in order to mitigate the thundering herd problem. This is bad, because the sum of multiple uniformly distributed random variables approaches a normal distribution, so the problem isn't mitigated as effectively as it would be with a single sleep. This change creates a single configurable anticongestion sleep. periodic will only sleep if at least one script requires it, and it will never sleep more than once per invocation. It also won't sleep if periodic was run interactively, fixing an unrelated longstanding bug. PR: 217055 PR: 210188 Reviewed by: cy MFC after: 3 weeks Differential Revision: https://reviews.freebsd.org/D10211 Changes: head/etc/defaults/periodic.conf head/etc/periodic/daily/480.leapfile-ntpd head/share/man/man5/periodic.conf.5 head/usr.sbin/periodic/periodic.sh A commit references this bug: Author: asomers Date: Mon Apr 24 15:29:18 UTC 2017 New revision: 317373 URL: https://svnweb.freebsd.org/changeset/base/317373 Log: MFC r316342, r316358 r316342: Consolidate random sleeps in periodic scripts Multiple periodic scripts sleep for a random amount of time in order to mitigate the thundering herd problem. This is bad, because the sum of multiple uniformly distributed random variables approaches a normal distribution, so the problem isn't mitigated as effectively as it would be with a single sleep. This change creates a single configurable anticongestion sleep. periodic will only sleep if at least one script requires it, and it will never sleep more than once per invocation. It also won't sleep if periodic was run interactively, fixing an unrelated longstanding bug. PR: 217055 PR: 210188 Reviewed by: cy MFC after: 3 weeks Differential Revision: https://reviews.freebsd.org/D10211 r316358: Fix man page typo from r316342 Reported by: rgrimes MFC after: 20 days X-MFC-With: 316342 Changes: _U stable/11/ stable/11/etc/defaults/periodic.conf stable/11/etc/periodic/daily/480.leapfile-ntpd stable/11/share/man/man5/periodic.conf.5 stable/11/usr.sbin/periodic/periodic.sh |
periodic daily, when invoked via terminal, can be witnessed sleeping (culprit being fetching port vulnerabilities, see below) # sh -x /usr/sbin/periodic daily + [ 1 -lt 1 ] + [ -r /etc/defaults/periodic.conf ] + . /etc/defaults/periodic.conf + periodic_conf_files='/etc/periodic.conf /etc/periodic.conf.local' + local_periodic=/usr/local/etc/periodic + daily_output=root + daily_show_success=YES + daily_show_info=YES + daily_show_badconfig=NO + daily_clean_disks_enable=NO + daily_clean_disks_files='[#,]* .#* a.out *.core *.CKP .emacs_[0-9]*' + daily_clean_disks_days=3 + daily_clean_disks_verbose=YES + daily_clean_tmps_enable=NO + daily_clean_tmps_dirs=/tmp + daily_clean_tmps_days=3 + daily_clean_tmps_ignore='.X*-lock .X11-unix .ICE-unix .font-unix .XIM-unix' + daily_clean_tmps_ignore='.X*-lock .X11-unix .ICE-unix .font-unix .XIM-unix quota.user quota.group .snap' + daily_clean_tmps_ignore='.X*-lock .X11-unix .ICE-unix .font-unix .XIM-unix quota.user quota.group .snap .sujournal' + daily_clean_tmps_verbose=YES + daily_clean_preserve_enable=YES + daily_clean_preserve_days=7 + daily_clean_preserve_verbose=YES + daily_clean_msgs_enable=YES + daily_clean_msgs_days='' + daily_clean_rwho_enable=YES + daily_clean_rwho_days=7 + daily_clean_rwho_verbose=YES + daily_clean_hoststat_enable=YES + daily_backup_passwd_enable=YES + daily_backup_aliases_enable=YES + daily_backup_pkgdb_enable=YES + daily_backup_pkgdb_dir=/var/backups + daily_calendar_enable=NO + daily_accounting_enable=YES + daily_accounting_compress=NO + daily_accounting_flags=-q + daily_accounting_save=3 + daily_news_expire_enable=YES + daily_status_disks_enable=YES + daily_status_disks_df_flags='-l -h' + daily_status_graid_enable=NO + daily_status_zfs_enable=NO + daily_status_zfs_zpool_list_enable=YES + daily_status_gmirror_enable=NO + daily_status_graid3_enable=NO + daily_status_gstripe_enable=NO + daily_status_gconcat_enable=NO + daily_status_network_enable=YES + daily_status_network_usedns=YES + daily_status_network_netstat_flags=-d + daily_status_rwho_enable=YES + daily_status_mailq_enable=YES + daily_status_mailq_shorten=NO + daily_status_include_submit_mailq=YES + daily_status_security_enable=YES + daily_status_security_inline=NO + daily_status_security_output=root + daily_status_mail_rejects_enable=YES + daily_status_mail_rejects_logs=3 + daily_status_mail_rejects_shorten=NO + daily_ntpd_leapfile_enable=NO + daily_ntpd_avoid_congestion=YES + daily_status_ntpd_enable=NO + daily_queuerun_enable=YES + daily_submit_queuerun=YES + daily_status_world_kernel=YES + daily_scrub_zfs_enable=NO + daily_scrub_zfs_pools='' + daily_scrub_zfs_default_threshold=35 + daily_local=/etc/daily.local + weekly_output=root + weekly_show_success=YES + weekly_show_info=YES + weekly_show_badconfig=NO + weekly_locate_enable=YES + weekly_whatis_enable=YES + weekly_catman_enable=NO + weekly_noid_enable=NO + weekly_noid_dirs=/ + weekly_status_security_enable=YES + weekly_status_security_inline=NO + weekly_status_security_output=root + weekly_local=/etc/weekly.local + monthly_output=root + monthly_show_success=YES + monthly_show_info=YES + monthly_show_badconfig=NO + monthly_accounting_enable=YES + monthly_status_security_enable=YES + monthly_status_security_inline=NO + monthly_status_security_output=root + monthly_local=/etc/monthly.local + security_status_logdir=/var/log + security_status_diff_flags='-b -u' + security_status_chksetuid_enable=YES + security_status_chksetuid_period=daily + security_status_neggrpperm_enable=YES + security_status_neggrpperm_period=daily + security_status_chkmounts_enable=YES + security_status_chkmounts_period=daily + security_status_noamd=NO + security_status_chkuid0_enable=YES + security_status_chkuid0_period=daily + security_status_passwdless_enable=YES + security_status_passwdless_period=daily + security_status_logincheck_enable=YES + security_status_logincheck_period=daily + security_status_chkportsum_enable=NO + security_status_chkportsum_period=daily + security_status_ipfwdenied_enable=YES + security_status_ipfwdenied_period=daily + security_status_ipfdenied_enable=YES + security_status_ipfdenied_period=daily + security_status_pfdenied_enable=YES + security_status_pfdenied_period=daily + security_status_ipfwlimit_enable=YES + security_status_ipfwlimit_period=daily + security_status_ipf6denied_enable=YES + security_status_ipf6denied_period=daily + security_status_kernelmsg_enable=YES + security_status_kernelmsg_period=daily + security_status_loginfail_enable=YES + security_status_loginfail_period=daily + security_status_tcpwrap_enable=YES + security_status_tcpwrap_period=daily + [ -z '' ] + source_periodic_confs_defined=yes + source_periodic_confs + local i sourced_files + sourced_files=:/etc/periodic.conf: + [ -r /etc/periodic.conf ] + sourced_files=:/etc/periodic.conf::/etc/periodic.conf.local: + [ -r /etc/periodic.conf.local ] + . /etc/periodic.conf.local + daily_backup_aliases_enable=NO + daily_clean_hoststat_enable=NO + daily_status_mail_rejects_enable=NO + daily_status_include_submit_mailq=NO + daily_submit_queuerun=NO + daily_output=/var/log/daily.log + security_output=/var/log/security + weekly_output=/var/log/weekly.log + monthly_output=/var/log/monthly.log + hostname + host=hephaistos.local + export host + [ daily != LOCKED ] + ret=0 + lockfile=/var/run/periodic.daily.lock + lockf -t 0 /var/run/periodic.daily.lock /bin/sh /usr/sbin/periodic LOCKED daily load: 0.31 cmd: find 1730 [running] 1.97r 0.13u 1.76s 18% 9952k load: 0.34 cmd: find 1730 [biord] 9.94r 0.33u 3.38s 17% 12760k load: 0.31 cmd: find 1730 [biord] 14.85r 0.36u 3.61s 12% 14020k load: 0.31 cmd: find 1730 [running] 17.80r 0.38u 3.85s 11% 14024k load: 0.31 cmd: find 1730 [biord] 18.51r 0.38u 3.90s 10% 14024k load: 0.28 cmd: sleep 1853 [nanslp] 33.72r 0.00u 0.00s 0% 1904k load: 0.28 cmd: sleep 1853 [nanslp] 35.25r 0.00u 0.00s 0% 1912k load: 0.22 cmd: sleep 1853 [nanslp] 46.57r 0.00u 0.00s 0% 1912k load: 0.22 cmd: sleep 1853 [nanslp] 46.95r 0.00u 0.00s 0% 1912k load: 0.19 cmd: sleep 1853 [nanslp] 149.61r 0.00u 0.00s 0% 1912k (the output at the end naturally comes from me hitting ^T) periodic line 118, invoking $file < /dev/null does probably not play too well with [ -t 0 ] tests to determine whether the script is being invoked from cron. One would assume that it's one of the /etc/periodic scripts that's sleeping. Interestingly, # grep -lri sleep . ./daily/480.leapfile-ntpd and given my daily_ntpd_leapfile_enable=NO, that shouldn't be doing anything. So let's look at /usr/local/etc/periodic : # grep -lri sleep . ./security/410.pkg-audit and indeed there we can see (L59+): # Random delay so the mirrors do not get slammed when run by periodic(8) if [ ! -t 0 ]; then sleep `jot -r 1 0 3600` fi As we've seen above, this script gets invoked with < /dev/null > tempfile, so [ -t 0 ] will always be false. But when being invoked from the terminal, one shouldn't sleep randomly. It doesn't make sense. The invocation time itself is random already, we're not coming from a cron ran periodic incantation.