Bug 265317 - sysutils/ipdbtools: add periodic weekly script for updating db
Summary: sysutils/ipdbtools: add periodic weekly script for updating db
Status: New
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Only Me
Assignee: freebsd-ports-bugs (Nobody)
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2022-07-19 15:45 UTC by Nick Hibma
Modified: 2022-07-24 20:57 UTC (History)
1 user (show)

See Also:
bugzilla: maintainer-feedback? (cyclaero)


Attachments
Periodic script and associated changes in Makefile and pkg-message (2.04 KB, patch)
2022-07-19 15:47 UTC, Nick Hibma
no flags Details | Diff
All files for port (1.82 KB, application/octet-stream)
2022-07-19 16:02 UTC, Nick Hibma
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Nick Hibma 2022-07-19 15:45:39 UTC

    
Comment 1 Nick Hibma 2022-07-19 15:47:19 UTC
Created attachment 235362 [details]
Periodic script and associated changes in Makefile and pkg-message

My attempt at fixing up the port for the change. Badly written and limited testing. YMMV
Comment 2 Nick Hibma 2022-07-19 16:02:14 UTC
Created attachment 235363 [details]
All files for port

This attachment contains all the files from the port in 1 tar file for your reference.
Comment 3 cyclaero 2022-07-24 16:39:35 UTC
(In reply to Nick Hibma from comment #1)

Hello, I am not convinced that this is a good idea.

   1.) ideally the upstream RIR’s are different for different regions.

   2.) while the RIR’s are usually reliable data providers, it happened to me in the past few years of using the ipdtools that the updating failed (for example, see this issue: https://github.com/cyclaero/ipdb/issues/1).

   3.) why make easy things complicated? What’s wrong with the cron job, which I suggested in the man file - s.: https://cyclaero.github.io/ipdb/#cronjob

   4.) I cannot find it written somewhere, but in my head the perhaps un written policy of the daily, weekly and monthly scripts is that it should not involve stuff from upstream - perhaps somebody more proficient in *nix system administration may want to vote in on this.

All said above dowe not mean „NO". Only, I want to see some better reasoning for the change.
Comment 4 Nick Hibma 2022-07-24 20:57:05 UTC
Thank you for your thoughtful comments!

wrt 1) You could randomize the pick if no arguments are given:

  MIRROR=$( (echo "ftp.lacnic.net"; echo "ftp.ripe.net" ) | sort -R | tail -1 )


wrt 2) sure, but that is more a case of replacing the target file only if the download succeeded.


wrt 3) certbot required the use of a cronjob as well, which was cumbersome and error prone; now this thing is included in the weekly mails it is trivial to set up, get right and get the result in the weekly mails.

Case in point on crontabs: Did you know that '%' is a special char in crontabs? I didn't...


wrt 4) There is some mitigation in the weekly scripts for that:

/etc/defaults/periodic.conf:

  # Max time to sleep to avoid causing congestion on download servers
  anticongestion_sleeptime=3600

In certbot weekly script (also in my weekly script as it is a copy of that script) a call to the 'anticongestion' function.

And: people copy&paste your crontab entry, so you get the same problem.