Updating e.g. net/seaweedfs to 3.19 requires using the new `M2T_GITHUB` env var to fetch repos. However this is insufficient for some repos, for example aws/aws-sdk-go-v2 returns: getting refs for aws/aws-sdk-go-v2: apis.get https://api.github.com/repos/aws/aws-sdk-go-v2/git/refs/tags: 502, It appears to be a limit *on the requested repo* as all the prior calls work, just the AWS ones fail. Re-running these requests from a different IP, with a different Github account's API key, hits the same limit. https://docs.github.com/en/rest/overview/resources-in-the-rest-api#rate-limiting > For requests to resources that belong to an enterprise account on GitHub.com, > GitHub Enterprise Cloud's rate limit applies, and the limit is 15,000 requests > per hour per repository. ... $ cd /usr/ports/net/seaweedfs $ sed -E -i '' -e 's/^PORTREV.*//' -e 's/2.81/3.19/' Makefile $ truncate -s 0 distinfo Makefile.deps $ make makesum $ env M2T_DEBUG=1 M2T_GITHUB=$YOU:TOKEN make gomod-vendor | tee Makefile.new ... cznic:strutil:5f824aabbbe044946f35921791abccfe3b1eafe1:cznic_strutil/vendor/modernc.org/strutil \ cznic:token:46658f20d51b8d0b6c4c551c363c8510c30c2a72:cznic_token/vendor/modernc.org/token # Errors found during processing: # error getting refs for aws/aws-sdk-go-v2: apis.get https://api.github.com/repos/aws/aws-sdk-go-v2/git/refs/tags: 502, body: { "message": "Server Error" } ... ``` Is this an issue with gomod-vendor, github api limits? I has asked github to clarify, will report back here.
Note that the error is 502, gateway error at github, and may not be an API limit directly.
feedback from github: > The error is due to a large number of tags on the repository in > question. We expect that a large number of tags is exceeding the > maximum object size allowed in the internal cache, so the tags are > likely being fetched from the repository; this is an expensive > operation that is likely contributing to the request being timed out. > > There's very little we can do about the various timeouts and limits so > we suggest instead focusing on the repository itself. The number of > tags in a repository typically increases over time, but we generally > recommend keeping the total number of references in a repository to a > few thousand. I'd assume some existing tags may no longer be required, > so it will be great to clean them up. Reducing the number of tags to > 2,000 or less is likely to yield the best results. there we go! I don't see anything we can do further here, and Dmitri already provided a neat improvement to the port I originally ran into this issue, so closing.