Bug 20498
| Summary: | All FreeBSD systems trigger massive late-night activity at the same times | ||
|---|---|---|---|
| Product: | Base System | Reporter: | brett <brett> |
| Component: | conf | Assignee: | Brian Somers <brian> |
| Status: | Closed FIXED | ||
| Severity: | Affects Only Me | ||
| Priority: | Normal | ||
| Version: | Unspecified | ||
| Hardware: | Any | ||
| OS: | Any | ||
|
Description
brett
2000-08-09 09:10:00 UTC
On Wed 2000-08-09 (01:09), brett@lariat.org wrote: > >Number: 20498 > >Category: conf > >Synopsis: All FreeBSD systems trigger massive late-night activity at the same times > >Description: > All FreeBSD systems, for a very long time, have come with a fixed /etc/crontab > which triggers system security and statistics-gathering scripts at the same > time on each system. A room full of FreeBSD machines is a sight and sound to > behold at this "witching hour." The sudden churning of the hard drives, and the > slowdown in the entire network (especially if the machines are mail, news, or > Web servers) can be quite dramatic. So can the power surge; we see the load > indicators on our UPSes peak at that time. > >How-To-Repeat: > > >Fix: > It'd be a good idea to randomize the times at which these scripts are > triggered at install time. The run times should be distributed within > a 3-hour period -- say, from 2 to 5 AM. This is a system's administrator's job, surely? A gratuitous change like this by default will confuse and probably irritate many. And I personally wouldn't like to have to document it. One possible solution is to add a 000.time-wait.sh script to your periodic/daily directory, which has "perl -e 'sleep int(rand(180))' in it (overkill, I know). You can change '180' to get a number from a variable in periodic.conf, for example. It would default to 0, or something. If you don't want _each_ invocation randmomized, then make the number you get from a variable in periodic.conf be the exact offset. Neil -- Neil Blakey-Milner Sunesi Clinical Systems nbm@mithrandr.moria.org At 05:56 AM 8/9/2000, Neil Blakey-Milner wrote: >This is a system's administrator's job, surely? Sysadmins already have a lot to do manually installing the OS. Why make their lives harder? >A gratuitous change >like this by default will confuse and probably irritate many. It's not gratuitous. It does not make sense to have EVERY FreeBSD server in the same time zone suddenly thrash at the same time! >And I >personally wouldn't like to have to document it. It'd be easy. Simply say that the scripts do a random delay (which could, perhaps, be turned off via a variable in rc.conf) and then continue. >One possible solution is to add a 000.time-wait.sh script to your >periodic/daily directory, which has "perl -e 'sleep int(rand(180))' in >it (overkill, I know). > >You can change '180' to get a number from a variable in periodic.conf, >for example. It would default to 0, or something. If you don't want >_each_ invocation randmomized, then make the number you get from a >variable in periodic.conf be the exact offset. Why not put that random delay right in the daily/weekly/monthly scripts? Or in /etc/crontab, where it's easy to spot? --Brett On Wed 2000-08-09 (13:45), Brett Glass wrote: > >This is a system's administrator's job, surely? > > Sysadmins already have a lot to do manually installing the OS. > Why make their lives harder? It's a policy decision. You have to make it one way or another. It is just as much work either way. > >A gratuitous change > >like this by default will confuse and probably irritate many. > > It's not gratuitous. It does not make sense to have EVERY > FreeBSD server in the same time zone suddenly thrash at the > same time! It's just as reasonable as having them go off on a random time in a three hour period. It's a personal preference. (for example, I know I begin to wonder what's up when one of my machines hasn't sent me mail by 2:13am) > >And I > >personally wouldn't like to have to document it. > > It'd be easy. Simply say that the scripts do a random delay > (which could, perhaps, be turned off via a variable in rc.conf) > and then continue. Or, "If you'd like your runs to be at a somewhat random time within three hours of 2am, you can set this variable...". (It would be in periodic.conf) > >One possible solution is to add a 000.time-wait.sh script to your > >periodic/daily directory, which has "perl -e 'sleep int(rand(180))' in > >it (overkill, I know). > > > >You can change '180' to get a number from a variable in periodic.conf, > >for example. It would default to 0, or something. If you don't want > >_each_ invocation randmomized, then make the number you get from a > >variable in periodic.conf be the exact offset. > > Why not put that random delay right in the daily/weekly/monthly scripts? > Or in /etc/crontab, where it's easy to spot? There is no "daily/weekly/monthly scripts". It is something run by periodic, which means that it takes, in numeric sorted order, all files of name "*.sh" which are executable, and runs them one after the other. A reasonable expectation of 000.time-wait.sh is that it'd be run first. mergemaster would want to eat this change whenever /etc/crontab updated. Very irritating. Neil -- Neil Blakey-Milner Sunesi Clinical Systems nbm@mithrandr.moria.org State Changed From-To: open->feedback The originator reckons that it should be easy to add something like "periodic_time_variant" to periodic.conf(8). I agree; the maintainer should look forward to patches from the originator. Responsible Changed From-To: freebsd-bugs->brian Over to the maintainer, who'll be interested in the patches. Responsible Changed From-To: brian->freebsd-bugs I don't believe it's appropriate to do this in periodic.conf. It should be done in the crontab file to remove the fact that a reboot between 2:00 and the actual start time would abort the daily run. State Changed From-To: feedback->closed In that case, unless Paul Vixie has some bright ideas, the status quo will have to do. On Thu, 10 Aug 2000 brian@FreeBSD.ORG wrote: > I don't believe it's appropriate to do this in periodic.conf. It > should be done in the crontab file to remove the fact that a > reboot between 2:00 and the actual start time would abort the > daily run. Good point. This seems to be a general problem (not just for periodic) and crontab would have the same difficulty, or? Timestamp files (eewww) would theoreticaly be a solution, but doesn't appeal to me practicaly. Not that I am for this feature as part of FreeBSD anyway... (just someone who reads -bugs, I'll crawl back into -questions now :) -Paul. State Changed
From-To: closed->open
I think it might be nice to have a mechanism whereby cron runs
maybe-periodic every 15 minutes between (say) 2:00 and 23:00. The
maybe-periodic script checks a touch file that indicates when the
last such periodic actually ran (say /var/db/periodic-{dai,week,month}ly.run.
If the file has a date of yesterday or before (or this day last week
or month depending on the periodic argument), the script picks a
random number between 1 and 24 (or 1 and 7 or 1 and 31) . If the
file is more than that number of hours (or days) old, the touch-file
is re-touched and the periodic run is invoked.
This resolves the random-start-time issue, and also handles systems
that are shut down every night.
I'll implement this if nobody objects.
Responsible Changed From-To: freebsd-bugs->brian I changed my mind. This can be resolved with a combination of cron and periodic changes > I don't believe it's appropriate to do this in periodic.conf. It should
> be done in the crontab file to remove the fact that a reboot between 2:00
> and the actual start time would abort the daily run.
For the OP, here's another idea: Change the crontab on every machine, so
instead of running /etc/periodic, run something like
echo periodic daily | at -m now + `jot -r 1 0 240` minutes
+ all systems have the same crontab
+ daily runs don't get lost in a reboot
+ all systems will fire up at a different time
State Changed From-To: open->closed On retrospect, I don't think it's practical to try to fix this. For large FreeBSD installations I think the only real way of avoiding this is to locally customise /etc/crontab |
