Bug 256145

Summary:	Corrupted distribution files when installing from Brazil FTP Servers
Product:	Services	Reporter:	Antonio Modesto <modesto916>
Component:	FTP/WWW Sites & Mirrors	Assignee:	FreeBSD Mirror Admin <mirror-admin>
Status:	Closed Not A Bug
Severity:	Affects Many People	CC:	lwhsu, philip
Priority:	---
Version:	unspecified
Hardware:	amd64
OS:	Any

Description Antonio Modesto 2021-05-25 11:41:33 UTC

Hello,

I noticed a strange problem when trying to upgrade my jails to 13.0-RELEASE. After downloading base.txz from ftp0.bra.freebsd.org, the extraction process is failing:

Example output:

# fetch 'ftp://ftp0.bra.freebsd.org/pub/FreeBSD/releases/amd64/13.0-RELEASE/base.txz'
# xzdec base.txz | tar --unlink -vxpJf - -C ${BASEDIR}
...
x ./usr/bin/unzstd
x ./usr/bin/kinit
x ./usr/bin/uudecode
x ./usr/bin/sort
x ./usr/bin/slc
x ./usr/bin/primes
x ./usr/bin/tset
x ./usr/bin/fmt
x ./usr/bin/clang-cpp
x ./usr/bin/ibv_srq_pingpong
x ./usr/bin/join
x ./usr/bin/ctags
x ./usr/bin/timeout
x ./usr/bin/bzip2recover
x ./usr/bin/resizewin
x ./usr/bin/ypmatch
x ./usr/bin/strip
x ./usr/bin/lldb: Lzma library error: Corrupted input data
tar: Error exit delayed from previous errors.


I tried the same process from different hosts but the problem stays the same. It also happens with 12.2-RELEASE. I forced the download from a mirror located in the US and it worked without problems. I believe there is some problem with the Brazilian mirrors.

An important detail is that ftp.freebsd.org points to ftp0.bra.freebsd.org when downloading from a host in Brazil, so it is difficult to specify another server, I had to point to a IP address of a foreign mirror to be able to download the files I needed.

Comment 1 Philip Paeps freebsd_committer

2021-05-25 14:18:47 UTC

Thanks for reporting this!

The Brazil mirror is the most recent one I set up.  I might have done something wrong.

Comment 2 Philip Paeps freebsd_committer

2021-05-25 14:35:09 UTC

I did some quick testing on the mirror.  I can run your command fine.  Comparing the checksums of base.txz with the checksum on ftp-master, they're also identical.

Could you please try two things:

1. Try the download over http instead of ftp (maybe something is messing with FTP?)
2. Please check the checksum of base.txz before trying to extract it

As far as I can tell, the mirror is correctly configured.

Comment 3 Michael Büker 2021-05-25 16:45:38 UTC

I just tried the same thing from Germany (deleting the file between the two runs):

$ fetch 'ftp://ftp0.bra.freebsd.org/pub/FreeBSD/releases/amd64/13.0-RELEASE/base.txz' && sha256 base.txz
SHA256 (base.txz) = be7ab3c21ca650b3763ad3605edd18c39d570abd40b0aad3058df64c3e38fb51

$ fetch http://ftp.freebsd.org/pub/FreeBSD/releases/amd64/13.0-RELEASE/base.txz && sha256 base.txz
SHA256 (base.txz) = be7ab3c21ca650b3763ad3605edd18c39d570abd40b0aad3058df64c3e38fb51

Maybe your ISP is messing with the unencrypted FTP data stream ...?

Comment 4 Antonio Modesto 2021-05-26 12:37:30 UTC

Sorry guys,


I think the problem is related to our ISP. For some reason the download finishes, the file size matches, but the md5 checksum does not match. It is really weird because the problem only happens with base.txz from the Brazilian FTP, the other distribution files have a correct checksum.

I have set up a mirror for our internal use to avoid this problem for a while. I appreciate your time and willingness to solve the problem.


Regards.

Comment 5 Li-Wen Hsu freebsd_committer

2021-05-26 14:37:03 UTC

(In reply to Antonio Modesto from comment #4)
Thanks for the update, BTW, we also provide https://download.freebsd.org/ which might help this situation.