Created attachment 156081 [details] patch Adam, As a followup to our yesterday conversation, this patch changes URLs to what MaxMind customer service recommended. They indeed fix the problem. Yuri
Created attachment 156085 [details] Updated patch Fixed an error code in case downloads fail.
Hi Yuri, I really appreciate you doing the research on this! The updated patch isn't necessary; the script already exits with return code 1 if either fetch fails. Past that though, my only reservation about committing this patch is that the URL that you provided, despite the fact that I can confirm that it works, it isn't listed on their website. They say to use the URL that we already use. Can you forward me the communication that you had with them wherein they advised you to use the new URL? Sorry for being a pain on this.
Yuri I am going to reject this PR. I spoke with MaxMind and they confirmed that, while the URL you provided does work, it is not the server from which they prefer the database be downloaded. They also indicated that they weren't aware of any specific reason why the current URL wouldn't work through Tor. That said, this PR will be crawled so the solution you proposed will be visible if people Google the problem in the future.
> The updated patch isn't necessary; the script already exits with return code 1 if either fetch fails. No, it doesn't pass an error out properly. Consider this script demo.sh: > #!/bin/sh > > fail() { > echo "failing" > return 1 > } > > succ() { > echo "succeeding" > return 0 > } > > fail > succ Run it: > $ ./demo.sh > failing > succeeding > [yuri@yuri ~/tmp]$ echo $? > 0 Part of it failed, yet it returns success code.
geoipupdate.sh traps error codes. [root@apnoea ~] tail -3 /usr/local/bin/geoipupdate.sh _fetch "FAILhttp://geolite.maxmind.com/download/geoip/database/GeoLiteCountry/GeoIP.dat.gz" GeoIP.dat _fetch "http://geolite.maxmind.com/download/geoip/database/GeoIPv6.dat.gz" GeoIPv6.dat [root@apnoea ~] geoipupdate.sh Fetching GeoIP.dat and GeoIPv6.dat... fetch: FAILhttp://geolite.maxmind.com/download/geoip/database/GeoLiteCountry/GeoIP.dat.gz: Invalid URL scheme GeoIP.dat download failed (1)[root@apnoea ~] echo $? 1
> geoipupdate.sh traps error codes. I see, ok.
Created attachment 156117 [details] Improved patch I hate to be too intrusive, but maybe you can consider this updated patch. It tries the second URL when the first one fails. I had to remove "set -e" to make some commands error-tolerant. I think this is about the best that can be done in this situation.
Sorry Yuri, but I can't commit that patch. The port needs to respect that MaxMind lists only one public server for obtaining the GeoIP database. Though the second server may work today, there's no guarantee that it will work next week, or that the database will be the same, or updated at the same time, on the second server. MaxMind indicated to me that they preferred that the database be downloaded from the listed location, so that location is what the GeoIP port needs to respect. That said, I'll do the next best thing, I'll let you override it with an environment variable. Fair enough?
A commit references this bug: Author: adamw Date: Thu Apr 30 04:28:29 UTC 2015 New revision: 385024 URL: https://svnweb.freebsd.org/changeset/ports/385024 Log: Allow the server for the GeoIP database to be altered by an environment variable. PR: 199768 While here, regenerate patch with 'make makepatch' to appease portlint. Changes: head/net/GeoIP/Makefile head/net/GeoIP/files/geoipupdate.sh.in head/net/GeoIP/files/patch-man__Makefile.am
> That said, I'll do the next best thing, I'll let you override it with an environment variable. Fair enough? Thanks.
For the record: http://geolite.maxmind.com is CloudFlare-based URL. CloudFlare requires all tor users to type captcha. This is the reason for the failure in the first place.
Adam, Would you oppose to the patch that, in case of download error, will check if this is coming from the tor network, and in in this case will use the alternative URL that is known to work for tor? This way only behavior in the currently failing case is improved, and nothing changes otherwise.
Sorry, Yuri, but only the download servers listed by MaxMind on their website will be in geoipupdate.sh.