Periodic scripts that hit external servers can cause a "thundering herd" problem, whereby many systems' periodic scripts run at the same time, and overload the server. To prevent this, such scripts should sleep for a random amount of time before hitting the external server. After r316432, there is a common anticongestion function to handle this. For an example of how to use it, see https://bugs.freebsd.org/bugzilla/attachment.cgi?id=181374&action=diff
While I see you've put some time, and thought into this, and I want to take nothing away from that. I can't help but wonder why the onus should be on FreeBSD. IOW this has always seemed like an administrators task to me. Don't you think? I mean, I've got clients with hundreds of servers, and I've always staggered them, time wise. For the periodic(8), and related tasks. Doesn't everyone, in this situation? In fact, Sendmail has employed a feature that could easily apply to this type of situation already -- retry after X-timeframe, and do so X-many times. To me, this approach seems just plain wrong. If this shows up in -CURRENT, I'm pulling it out. It's my server, and it should work on my schedule. Don't you think? In the end. If this is the way FreeBSD insists on doing it. I'm not going to fight it. So I guess you can consider this a reluctant approval. ;-) Sincerely, disappointed :-(
Sure, sysadmins have been doing this for a long time. But isn't it better that they don't have to? I mean, for years sysadmins have also been building their own software through ports, but now they usually don't have to, thanks to pkgng. BTW, you can disable this behavior by putting "anticongestion_sleeptime=0" in your /etc/periodic.conf file.
(In reply to Alan Somers from comment #2) > Sure, sysadmins have been doing this for a long time. But isn't it better > that they don't have to? I mean, for years sysadmins have also been > building their own software through ports, but now they usually don't have > to, thanks to pkgng. I *still* build my software. :-) IOW one persons idea of the "right" choice of options, are not necessarily the "right" choice for another person. :-) > > BTW, you can disable this behavior by putting "anticongestion_sleeptime=0" > in your /etc/periodic.conf file. Thanks for the hint! :-) I still think it's the wrong way look at this situation. But I'll [mostly] stop bitching about it now. ;-) Thank you, Alan, for all the time you've put into this, and taking the *extra* time to respond -- *greatly* appreciated! --Chris
While I have/employ what I believe to be the "correct" policy regarding this type of situation. I'll yield to alan's request. As I *also* except the fact I'm not the *only* one to use this. :-) See also bug 218449 for further commentary on this. :-) Thanks, and consider this my approval of the purposed patch/diff. --Chris
Pending patch attachment. Please set flag: maintainer-approval ? <maintainer-email> on it, so they may approve. Reporter is committer, assign accordingly. @Alan, if you need ports approval, just let me know and I'll review
Created attachment 191232 [details] Add an anticongestion sleep to boxbackup's monthly periodic job Here's a patch that covers both PR 218448 and PR 218449, for your approval. I haven't tested it at all.
A commit references this bug: Author: asomers Date: Mon May 21 16:02:28 UTC 2018 New revision: 470559 URL: https://svnweb.freebsd.org/changeset/ports/470559 Log: sysutils/boxbackup*: use anticongestion sleep in the periodic job Use an anticongestion sleep in the periodic job to prevent the thundering herd problem when many systems' cron jobs fire off at once. PR: 218448 PR: 218449 Approved by: portmaster@bsdforge.com (maintainer) Changes: head/sysutils/boxbackup/Makefile head/sysutils/boxbackup/files/999.boxbackup.in head/sysutils/boxbackup-devel/Makefile head/sysutils/boxbackup-devel/files/999.boxbackup.in