Hello, https://git.freebsd.org/src.git times out over ipv6 and works normally over ipv4. In comparison, https://gitlab.com/FreeBSD/freebsd-src.git over ipv6 works normally. The following output of git clone is from an openbsd 7.4 machine which was created to exclude local freebsd from being an issue. radish$ export GIT_TRACE_PACKET=1 radish$ export GIT_TRACE=1 radish$ export GIT_CURL_VERBOSE=1 radish$ time git clone -6 https://git.freebsd.org/src.git 21:26:48.654685 git.c:463 trace: built-in: git clone -6 https://git.freebsd.org/src.git Cloning into 'src'... 21:26:50.268069 run-command.c:659 trace: run_command: git remote-https origin https://git.freebsd.org/src.git 21:26:50.291055 git.c:749 trace: exec: git-remote-https origin https://git.freebsd.org/src.git 21:26:50.292566 run-command.c:659 trace: run_command: git-remote-https origin https://git.freebsd.org/src.git 21:26:50.349439 http.c:820 == Info: Couldn't find host git.freebsd.org in the .netrc file; using defaults 21:26:50.511109 http.c:820 == Info: Host git.freebsd.org:443 was resolved. 21:26:50.511563 http.c:820 == Info: IPv6: 2604:1380:4091:a001::24ca:1, 2a02:80:0:3ffd::24ca:1 21:26:50.511652 http.c:820 == Info: IPv4: (none) 21:26:50.512185 http.c:820 == Info: Trying [2604:1380:4091:a001::24ca:1]:443... 21:26:50.534006 http.c:820 == Info: Connected to git.freebsd.org (2604:1380:4091:a001::24ca:1) port 443 21:26:50.536437 http.c:820 == Info: ALPN: curl offers h2,http/1.1 21:26:50.540908 http.c:820 == Info: TLSv1.3 (OUT), TLS handshake, Client hello (1): 21:26:50.656884 http.c:820 == Info: CAfile: /etc/ssl/cert.pem 21:26:50.656983 http.c:820 == Info: CApath: none 21:31:50.381351 http.c:820 == Info: SSL connection timeout 21:31:50.381635 http.c:820 == Info: Closing connection fatal: unable to access 'https://git.freebsd.org/src.git/': SSL connection timeout 5m03.52s real 0m00.03s user 0m00.07s system radish$ ### radish$ uname -a OpenBSD radish 7.4 GENERIC.MP#2 arm64 radish$ radish$ host git.freebsd.org git.freebsd.org is an alias for gitmir.geo.freebsd.org. gitmir.geo.freebsd.org has address 147.28.184.44 gitmir.geo.freebsd.org has address 85.30.190.139 gitmir.geo.freebsd.org has IPv6 address 2604:1380:4091:a001::24ca:1 gitmir.geo.freebsd.org has IPv6 address 2a02:80:0:3ffd::24ca:1 gitmir.geo.freebsd.org mail is handled by 0 . radish$ radish$ ping6 git.freebsd.org ping6: Warning: gitmir.geo.freebsd.org has multiple addresses; using 2604:1380:4091:a001::24ca:1 PING gitmir.geo.freebsd.org (2604:1380:4091:a001::24ca:1): 56 data bytes 64 bytes from 2604:1380:4091:a001::24ca:1: icmp_seq=0 hlim=252 time=21.579 ms 64 bytes from 2604:1380:4091:a001::24ca:1: icmp_seq=1 hlim=252 time=21.204 ms 64 bytes from 2604:1380:4091:a001::24ca:1: icmp_seq=2 hlim=252 time=21.006 ms 64 bytes from 2604:1380:4091:a001::24ca:1: icmp_seq=3 hlim=252 time=20.947 ms 64 bytes from 2604:1380:4091:a001::24ca:1: icmp_seq=4 hlim=252 time=20.974 ms ^C --- gitmir.geo.freebsd.org ping statistics --- 5 packets transmitted, 5 packets received, 0.0% packet loss round-trip min/avg/max/std-dev = 20.947/21.142/21.579/0.237 ms radish$ traceroute6 2a02:80:0:3ffd::24ca:1 traceroute6 to 2a02:80:0:3ffd::24ca:1 (2a02:80:0:3ffd::24ca:1), 64 hops max, 60 byte packets 1 2a02:8010:[redacted]:1 (2a02:8010:[redacted]:1) 0.874 ms 0.629 ms 0.476 ms 2 lo-0.cor1.lond1.ptn.zen.net.uk (2a02:8010::100) 7.34 ms 7.381 ms 31.434 ms 3 ae-8.p1.thn-lon.zen.net.uk (2a02:8010:0:700::3a) 5.706 ms 5.629 ms 5.819 ms 4 * * * 5 ldn-bb1-v6.ip.twelve99.net (2001:2034:1:7a::1) 6.128 ms 5.926 ms 5.811 ms 6 hbg-bb3-v6.ip.twelve99.net (2001:2034:1:6f::1) 18.516 ms * * 7 s-bb1-v6.ip.twelve99.net (2001:2034:1:c4::1) 35.105 ms 35.213 ms s-bb2-v6.ip.twelve99.net (2001:2034:1:c5::1) 28.671 ms 8 * * * 9 * * * 10 2a02:80:0:3ffc::3 (2a02:80:0:3ffc::3) 43.194 ms 39.916 ms 39.804 ms 11 2a02:80:0:3ffc::3 (2a02:80:0:3ffc::3) 39.489 ms 39.269 ms 42.592 ms 12 gitmir.sjb.freebsd.org (2a02:80:0:3ffd::24ca:1) 46.085 ms 42.736 ms 42.582 ms radish$ traceroute6 2604:1380:4091:a001::24ca:1 traceroute6 to 2604:1380:4091:a001::24ca:1 (2604:1380:4091:a001::24ca:1), 64 hops max, 60 byte packets 1 2a02:8010:[redacted]:1 (2a02:8010:[redacted]:1) 0.917 ms 0.76 ms 0.539 ms 2 lo-0.cor1.lond1.ptn.zen.net.uk (2a02:8010::100) 8.926 ms 7.378 ms 9.098 ms 3 ae-8.p1.thn-lon.zen.net.uk (2a02:8010:0:700::3a) 5.982 ms 5.686 ms 5.584 ms 4 linx-lon2.ae211.bor01.ld8.eu.equinix.net (2001:7f8:4:1::3dd6:1) 5.677 ms 6.867 ms 5.478 ms 5 * * * 6 2001:4d78:fe04::8:6a (2001:4d78:fe04::8:6a) 20.882 ms 20.932 ms 28.002 ms 7 * * * 8 * * * 9 * * * 10 * * * 11 gitmir.fra.freebsd.org (2604:1380:4091:a001::24ca:1) 21.046 ms 21.015 ms 20.978 ms radish$
A speedtest from my location to Secaucus, NY on ipv6: ✓ Test Server: [Custom] [2604:a00:50:14::2]:8080 ✓ Latency: 39.7657ms Jitter: 10.913363ms Min: 36.060171ms Max: 72.505346ms ✓ Download: 50.04Mbps (used: 59.65MB) (latency: 37ms jitter: 1ms min: 36ms max: 39ms) ✓ Upload: 13.07Mbps (used: 15.58MB) (latency: 40ms jitter: 10ms min: 36ms max: 72ms) A workaround at this time for things like freebsd-update that don't have -4 or -6 switches is to set ip6addrctl_policy="ipv4" and reboot. Otherwise freebsd-update won't work in a dual stack environment
This works find from here and from a couple of other places on the internet. I suspect a PMTU issue on your end, or somewhere along your path. Could you please capture tcpdump output from this: curl -vk -H 'Host git.freebsd.org' https://gitmir.fra.freebsd.org/ Pay close attention to the MSS values. This mirror has been running with MTU=9000 for a couple of months -- since bringing up the newer site in Sweden. I should bring it back to MTU=1500.
Aside: there is no freebsd-update mirror in Frankfurt. If you're seeing timeouts on freebsd-update too, that indicates a wider IPv6 issue.
Created attachment 249088 [details] tcpdump output as plain txt
(In reply to Philip Paeps from comment #2) (In reply to Philip Paeps from comment #2) Hi, I had to use curl -6 because of the modification to rc.conf explained previously. I have attached the tcpdump output (as plain txt rather than a pcap) In the curl terminal, this was the result: curl -6 -vk -H 'Host git.freebsd.org' https://gitmir.fra.freebsd.org/ * Host gitmir.fra.freebsd.org:443 was resolved. * IPv6: 2604:1380:4091:a001::24ca:1 * IPv4: (none) * Trying [2604:1380:4091:a001::24ca:1]:443... * Connected to gitmir.fra.freebsd.org (2604:1380:4091:a001::24ca:1) port 443 * ALPN: curl offers h2,http/1.1 * TLSv1.3 (OUT), TLS handshake, Client hello (1): * Recv failure: Connection reset by peer * OpenSSL SSL_connect: Connection reset by peer in connection to gitmir.fra.freebsd.org:443 * Closing connection curl: (35) Recv failure: Connection reset by peer It seems I send with a MSS value of 1440 and the site replies with a MSS of 8940.
Yeah. As I expected: Path MTU Discovery fail. Something along the path is dropping ICMPv6 "Packet Too Big" messages. If you can control that machine, you can fix the problem before I get around to lowering the MTU on our end. ;-) Thanks for letting us know!
(In reply to Philip Paeps from comment #7) it's working now :D TYVM I'll need to reboot my machine to test freebsd-update, not sure if ipaddrctl can apply changes on the fly
mss is now 1440 on update2 so freebsd-update now works TYVM
I didn't actually change anything, but glad to hear that whatever middlebox was broken was fixed. ;-) I'll keep this bug open until I get around to lowering the MTU on our Frankfurt mirror though. Jumbograms on the internet are not a recipe for success.
spooky! I changed nothing here lol Now the tcpdump says the packets gitmir is sending are mss 1440, which works great
(In reply to Philip Paeps from comment #7) Hi, $ host git.freebsd.org | grep IPv6 gitmir.geo.freebsd.org has IPv6 address 2a02:80:0:3ffd::24ca:1 gitmir.geo.freebsd.org has IPv6 address 2604:1380:4091:a001::24ca:1 $ Both have mss of 8940. 2604:1380:4091:a001::24ca:1 works 2a02:80:0:3ffd::24ca:1 doesn't it appears that it's only gitmir.sjb.freebsd.org which has the problem
Well, now the middlebox is interfering with your traffic to our mirror in Sweden. ;-) I finally got around to starting a mirror refresh this morning, and taking the opportunity to lower the MTUs of the two EU mirrors. Both mirrors are back to MTU=1500.
confirmed it works, TYVM :D