Bug 218448

Summary: sysutils/boxbackup: periodic script needs anticongestion sleep
Product: Ports & Packages Reporter: Alan Somers <asomers>
Component: Individual Port(s)Assignee: Alan Somers <asomers>
Status: Closed FIXED    
Severity: Affects Some People CC: portmaster
Priority: --- Keywords: feature, needs-patch
Version: LatestFlags: portmaster: maintainer-feedback+
Hardware: Any   
OS: Any   
See Also: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=218449
Attachments:
Description Flags
Add an anticongestion sleep to boxbackup's monthly periodic job portmaster: maintainer-approval+

Description Alan Somers freebsd_committer freebsd_triage 2017-04-07 03:03:03 UTC 
Periodic scripts that hit external servers can cause a "thundering herd" problem, whereby many systems' periodic scripts run at the same time, and overload the server.  To prevent this, such scripts should sleep for a random amount of time before hitting the external server.

After r316432, there is a common anticongestion function to handle this.  For an example of how to use it, see
https://bugs.freebsd.org/bugzilla/attachment.cgi?id=181374&action=diff
Comment 1 Chris Hutchinson 2017-04-07 04:41:08 UTC 
While I see you've put some time, and thought into this,
and I want to take nothing away from that.
I can't help but wonder why the onus should be on FreeBSD.
IOW this has always seemed like an administrators task to me.
Don't you think? I mean, I've got clients with hundreds of
servers, and I've always staggered them, time wise. For the
periodic(8), and related tasks. Doesn't everyone, in this
situation? In fact, Sendmail has employed a feature that
could easily apply to this type of situation already --
retry after X-timeframe, and do so X-many times.

To me, this approach seems just plain wrong.
If this shows up in -CURRENT, I'm pulling it out. It's
my server, and it should work on my schedule. Don't
you think?

In the end. If this is the way FreeBSD insists on doing it.
I'm not going to fight it. So I guess you can consider this
a reluctant approval. ;-)

Sincerely,
 disappointed :-(
Comment 2 Alan Somers freebsd_committer freebsd_triage 2017-04-07 14:10:19 UTC 
Sure, sysadmins have been doing this for a long time.  But isn't it better that they don't have to?  I mean, for years sysadmins have also been building their own software through ports, but now they usually don't have to, thanks to pkgng.

BTW, you can disable this behavior by putting "anticongestion_sleeptime=0" in your /etc/periodic.conf file.
Comment 3 Chris Hutchinson 2017-04-07 15:03:39 UTC 
(In reply to Alan Somers from comment #2)
> Sure, sysadmins have been doing this for a long time.  But isn't it better
> that they don't have to?  I mean, for years sysadmins have also been
> building their own software through ports, but now they usually don't have
> to, thanks to pkgng.
I *still* build my software. :-)
IOW one persons idea of the "right" choice of options, are not necessarily
the "right" choice for another person. :-)
> 
> BTW, you can disable this behavior by putting "anticongestion_sleeptime=0"
> in your /etc/periodic.conf file.
Thanks for the hint! :-)

I still think it's the wrong way look at this situation. But I'll
[mostly] stop bitching about it now. ;-)

Thank you, Alan, for all the time you've put into this, and taking
the *extra* time to respond -- *greatly* appreciated!

--Chris
Comment 4 Chris Hutchinson 2018-02-24 02:14:37 UTC 
While I have/employ what I believe to be the "correct" policy
regarding this type of situation. I'll yield to alan's request.
As I *also* except the fact I'm not the *only* one to use this. :-)

See also bug 218449 for further commentary on this. :-)

Thanks, and consider this my approval of the purposed patch/diff.

--Chris
Comment 5 Kubilay Kocak freebsd_committer freebsd_triage 2018-02-24 04:01:02 UTC 
Pending patch attachment. Please set flag: maintainer-approval ? <maintainer-email> on it, so they may approve.

Reporter is committer, assign accordingly.

@Alan, if you need ports approval, just let me know and I'll review
Comment 6 Alan Somers freebsd_committer freebsd_triage 2018-03-05 20:15:18 UTC 
Created attachment 191232 [details]
Add an anticongestion sleep to boxbackup's monthly periodic job

Here's a patch that covers both PR 218448 and PR 218449, for your approval.  I haven't tested it at all.
Comment 7 commit-hook freebsd_committer freebsd_triage 2018-05-21 16:02:57 UTC 
A commit references this bug:

Author: asomers
Date: Mon May 21 16:02:28 UTC 2018
New revision: 470559
URL: https://svnweb.freebsd.org/changeset/ports/470559

Log:
  sysutils/boxbackup*: use anticongestion sleep in the periodic job

  Use an anticongestion sleep in the periodic job to prevent the thundering
  herd problem when many systems' cron jobs fire off at once.

  PR:		218448
  PR:		218449
  Approved by:	portmaster@bsdforge.com (maintainer)

Changes:
  head/sysutils/boxbackup/Makefile
  head/sysutils/boxbackup/files/999.boxbackup.in
  head/sysutils/boxbackup-devel/Makefile
  head/sysutils/boxbackup-devel/files/999.boxbackup.in