Bug 253814 - graphics/oidn: distinfo looks incorrect
Summary: graphics/oidn: distinfo looks incorrect
Status: Closed FIXED
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Some People
Assignee: Yuri Victorovich
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2021-02-24 13:43 UTC by Mamoru IWAKI
Modified: 2024-02-19 12:29 UTC (History)
2 users (show)

See Also:
bugzilla: maintainer-feedback? (yuri)


Attachments
errornously downloaded file (5.24 KB, application/gzip)
2024-02-17 15:57 UTC, Mamoru IWAKI
no flags Details
change "--format=bsdtar" to "--format=ustar" in Makefile (704 bytes, patch)
2024-02-18 13:43 UTC, Mamoru IWAKI
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Mamoru IWAKI 2021-02-24 13:43:16 UTC
Hi,

Re: # $FreeBSD: head/graphics/oidn/Makefile 566074 2021-02-19 15:14:46Z martymac $

I'm on FreeBSD dhcp79.tmnetwork.local 12.2-STABLE FreeBSD 12.2-STABLE r369342 MYGEN  amd64.  

Building of this port stopped after mismatched information written in distinfo.  
The distinfo should be remade by 'make makesum' at /usr/ports/graphics/oidn.  
The updated distinfo looks work well, anyway.  

Cheers
Comment 1 Yuri Victorovich freebsd_committer freebsd_triage 2021-02-24 19:30:02 UTC
(In reply to 1wkmmr from comment #0)

Hi 1wkmmr@gmail.com,

I can't reproduce this.

This port has a custom fetch because it has to fetch large LFS files, but 'make fetch' succeeds as of r566074.

Could you please try again? Otherwise, is there anything special about your system?


Yuri
Comment 2 Mamoru IWAKI 2021-02-25 05:45:12 UTC
(In reply to Yuri Victorovich from comment #1)

Hi, Yuri,

Thank you for your check.  
I know that 'make fetch' finishes successfully.  
Please check whether 'make checksum' would be successful.  
The following is a result of the above investigation.  

#I'm not sure, but wondering that the archive file in upstream 
#had changed with the same tag, without any notification, sometime.  

Cheers


root@dhcp79:/usr/ports/graphics/oidn # make fetch
===>  License APACHE20 accepted by the user
===>   oidn-1.3.0_1 depends on file: /usr/local/sbin/pkg - found
===>   oidn-1.3.0_1 depends on executable: git - found
===>   oidn-1.3.0_1 depends on executable: git-lfs - found
===> Fetching all distfiles required by oidn-1.3.0_1 for building
root@dhcp79:/usr/ports/graphics/oidn # make checksum
===>  License APACHE20 accepted by the user
===>   oidn-1.3.0_1 depends on file: /usr/local/sbin/pkg - found
===>   oidn-1.3.0_1 depends on executable: git - found
===>   oidn-1.3.0_1 depends on executable: git-lfs - found
===> Fetching all distfiles required by oidn-1.3.0_1 for building
=> SHA256 Checksum mismatch for oidn-weights-59bad6bb6344f8fb8205772df3f795c2dc72e23b.tar.gz.
=> SHA256 Checksum OK for OpenImageDenoise-oidn-v1.3.0_GH0.tar.gz.
=> SHA256 Checksum OK for OpenImageDenoise-mkl-dnn-eb3e967_GH0.tar.gz.
===>  Refetch for 1 more times files:  oidn-weights-59bad6bb6344f8fb8205772df3f795c2dc72e23b.tar.gz
===>  License APACHE20 accepted by the user
===>   oidn-1.3.0_1 depends on file: /usr/local/sbin/pkg - found
===>   oidn-1.3.0_1 depends on executable: git - found
===>   oidn-1.3.0_1 depends on executable: git-lfs - found
==> Fetching the oidn/weights subproject with large LFS files
Downloading rt_hdr.tza (3.7 MB)
Downloading rt_hdr_alb.tza (3.7 MB)
Downloading rt_hdr_alb_nrm.tza (3.7 MB)
Downloading rt_ldr.tza (3.7 MB)
Downloading rt_ldr_alb.tza (3.7 MB)
Downloading rt_ldr_alb_nrm.tza (3.7 MB)
Downloading rtlightmap_dir.tza (3.7 MB)
Downloading rtlightmap_hdr.tza (3.7 MB)
=> oidn-weights-59bad6bb6344f8fb8205772df3f795c2dc72e23b.tar.gz doesn't seem to exist in /usr/ports/distfiles/.
=> Attempting to fetch http://freebsd.org/oidn-weights-59bad6bb6344f8fb8205772df3f795c2dc72e23b.tar.gz
fetch: http://freebsd.org/oidn-weights-59bad6bb6344f8fb8205772df3f795c2dc72e23b.tar.gz: Not Found
=> Attempting to fetch http://distcache.FreeBSD.org/ports-distfiles/oidn-weights-59bad6bb6344f8fb8205772df3f795c2dc72e23b.tar.gz
fetch: http://distcache.FreeBSD.org/ports-distfiles/oidn-weights-59bad6bb6344f8fb8205772df3f795c2dc72e23b.tar.gz: Not Found
=> Couldn't fetch it - please try to retrieve this
=> port manually into /usr/ports/distfiles/ and try again.
*** Error code 1

Stop.
make[1]: stopped in /usr/ports/graphics/oidn
*** Error code 1

Stop.
make: stopped in /usr/ports/graphics/oidn
root@dhcp79:/usr/ports/graphics/oidn #
Comment 3 Yuri Victorovich freebsd_committer freebsd_triage 2021-02-25 05:52:02 UTC
(In reply to 1wkmmr from comment #2)

'make checksum' also succeeds for me:

> $ make fetch
> ===>  License APACHE20 accepted by the user
> ===>   oidn-1.3.0_1 depends on file: /usr/local/sbin/pkg - found
> ===>   oidn-1.3.0_1 depends on executable: git - found
> ===>   oidn-1.3.0_1 depends on executable: git-lfs - found
> ===> Fetching all distfiles required by oidn-1.3.0_1 for building
> $ make checksum
> ===>  License APACHE20 accepted by the user
> ===>   oidn-1.3.0_1 depends on file: /usr/local/sbin/pkg - found
> ===>   oidn-1.3.0_1 depends on executable: git - found
> ===>   oidn-1.3.0_1 depends on executable: git-lfs - found
> ===> Fetching all distfiles required by oidn-1.3.0_1 for building
> => SHA256 Checksum OK for oidn-weights-59bad6bb6344f8fb8205772df3f795c2dc72e23b.tar.gz.
> => SHA256 Checksum OK for OpenImageDenoise-oidn-v1.3.0_GH0.tar.gz.
> => SHA256 Checksum OK for OpenImageDenoise-mkl-dnn-eb3e967_GH0.tar.gz.
Comment 4 Mamoru IWAKI 2021-02-25 07:17:16 UTC
(In reply to Yuri Victorovich from comment #3)

Hi, Yuri,

Thank you for your checking.  

Sorry for this long reply with more investigation.  

#=#=#=#=#=#=#=#=#=#=#=#=#=#=#=#
Let me show you information about distinfo and downloaded file in my environment.  
The 2nd and 3rd lines in distinfo are mismatched with the downloaded ones.  


root@dhcp79:~ # cat /usr/ports/graphics/oidn/distinfo
TIMESTAMP = 1612495835
SHA256 (oidn-weights-59bad6bb6344f8fb8205772df3f795c2dc72e23b.tar.gz) = 169e25c8a49e13ea5dd94d7bd97feb4bc45247f0532a86d2b46d6ac4fd15d3e1
SIZE (oidn-weights-59bad6bb6344f8fb8205772df3f795c2dc72e23b.tar.gz) = 27410305
SHA256 (OpenImageDenoise-oidn-v1.3.0_GH0.tar.gz) = a6b30d7aa3ea81490431dba823dab475daad73a7124940560624742126db7274
SIZE (OpenImageDenoise-oidn-v1.3.0_GH0.tar.gz) = 3443375
SHA256 (OpenImageDenoise-mkl-dnn-eb3e967_GH0.tar.gz) = 5a46df10204a61daab19e8d1c01a074c1c2e98d7bb5303b13a4c7c592cc77b4f
SIZE (OpenImageDenoise-mkl-dnn-eb3e967_GH0.tar.gz) = 5795997
root@dhcp79:~ # ls -l /usr/ports/distfiles/oidn-weights-59bad6bb6344f8fb8205772df3f795c2dc72e23b.tar.gz
-rw-r--r--  1 root  wheel  27410393 Feb 25 14:34 /usr/ports/distfiles/oidn-weights-59bad6bb6344f8fb8205772df3f795c2dc72e23b.tar.gz
root@dhcp79:~ # sha256 /usr/ports/distfiles/oidn-weights-59bad6bb6344f8fb8205772df3f795c2dc72e23b.tar.gz
SHA256 (/usr/ports/distfiles/oidn-weights-59bad6bb6344f8fb8205772df3f795c2dc72e23b.tar.gz) = be1e929fb01bd2a8f1c0a0eabad7c050cb9fb20849fdb2c8231fb12baf860b9c
root@dhcp79:~ #


#=#=#=#=#=#=#=#=#=#=#=#=#=#=#=#
How about 'make distclean' first before 'make fetch' and 'make checksum'?
I got the following output with failure in my environment.  


root@dhcp79:/usr/ports/graphics/oidn # make distclean
===>  Cleaning for oidn-1.3.0_1
===>  Deleting distfiles for oidn-1.3.0_1
root@dhcp79:/usr/ports/graphics/oidn # make fetch
===>  License APACHE20 accepted by the user
===>   oidn-1.3.0_1 depends on file: /usr/local/sbin/pkg - found
===>   oidn-1.3.0_1 depends on executable: git - found
===>   oidn-1.3.0_1 depends on executable: git-lfs - found
==> Fetching the oidn/weights subproject with large LFS files
Downloading rt_hdr.tza (3.7 MB)
Downloading rt_hdr_alb.tza (3.7 MB)
Downloading rt_hdr_alb_nrm.tza (3.7 MB)
Downloading rt_ldr.tza (3.7 MB)
Downloading rt_ldr_alb.tza (3.7 MB)
Downloading rt_ldr_alb_nrm.tza (3.7 MB)
Downloading rtlightmap_dir.tza (3.7 MB)
Downloading rtlightmap_hdr.tza (3.7 MB)
=> OpenImageDenoise-oidn-v1.3.0_GH0.tar.gz doesn't seem to exist in /usr/ports/distfiles/.
=> Attempting to fetch https://codeload.github.com/OpenImageDenoise/oidn/tar.gz/v1.3.0?dummy=/OpenImageDenoise-oidn-v1.3.0_GH0.tar.gz
fetch: https://codeload.github.com/OpenImageDenoise/oidn/tar.gz/v1.3.0?dummy=/OpenImageDenoise-oidn-v1.3.0_GH0.tar.gz: size unknown
fetch: https://codeload.github.com/OpenImageDenoise/oidn/tar.gz/v1.3.0?dummy=/OpenImageDenoise-oidn-v1.3.0_GH0.tar.gz: size of remote file is not known
OpenImageDenoise-oidn-v1.3.0_GH0.tar.gz               3362 kB 3210 kBps    01s
=> OpenImageDenoise-mkl-dnn-eb3e967_GH0.tar.gz doesn't seem to exist in /usr/ports/distfiles/.
=> Attempting to fetch https://codeload.github.com/OpenImageDenoise/mkl-dnn/tar.gz/eb3e967?dummy=/OpenImageDenoise-mkl-dnn-eb3e967_GH0.tar.gz
fetch: https://codeload.github.com/OpenImageDenoise/mkl-dnn/tar.gz/eb3e967?dummy=/OpenImageDenoise-mkl-dnn-eb3e967_GH0.tar.gz: size unknown
fetch: https://codeload.github.com/OpenImageDenoise/mkl-dnn/tar.gz/eb3e967?dummy=/OpenImageDenoise-mkl-dnn-eb3e967_GH0.tar.gz: size of remote file is not known
OpenImageDenoise-mkl-dnn-eb3e967_GH0.tar.gz           5660 kB 8622 kBps    00s
===> Fetching all distfiles required by oidn-1.3.0_1 for building
root@dhcp79:/usr/ports/graphics/oidn # make checksum
===>  License APACHE20 accepted by the user
===>   oidn-1.3.0_1 depends on file: /usr/local/sbin/pkg - found
===>   oidn-1.3.0_1 depends on executable: git - found
===>   oidn-1.3.0_1 depends on executable: git-lfs - found
===> Fetching all distfiles required by oidn-1.3.0_1 for building
=> SHA256 Checksum mismatch for oidn-weights-59bad6bb6344f8fb8205772df3f795c2dc72e23b.tar.gz.
=> SHA256 Checksum OK for OpenImageDenoise-oidn-v1.3.0_GH0.tar.gz.
=> SHA256 Checksum OK for OpenImageDenoise-mkl-dnn-eb3e967_GH0.tar.gz.
===>  Refetch for 1 more times files:  oidn-weights-59bad6bb6344f8fb8205772df3f795c2dc72e23b.tar.gz
===>  License APACHE20 accepted by the user
===>   oidn-1.3.0_1 depends on file: /usr/local/sbin/pkg - found
===>   oidn-1.3.0_1 depends on executable: git - found
===>   oidn-1.3.0_1 depends on executable: git-lfs - found
==> Fetching the oidn/weights subproject with large LFS files
Downloading rt_hdr.tza (3.7 MB)
Downloading rt_hdr_alb.tza (3.7 MB)
Downloading rt_hdr_alb_nrm.tza (3.7 MB)
Downloading rt_ldr.tza (3.7 MB)
Downloading rt_ldr_alb.tza (3.7 MB)
Downloading rt_ldr_alb_nrm.tza (3.7 MB)
Downloading rtlightmap_dir.tza (3.7 MB)
Downloading rtlightmap_hdr.tza (3.7 MB)
=> oidn-weights-59bad6bb6344f8fb8205772df3f795c2dc72e23b.tar.gz doesn't seem to exist in /usr/ports/distfiles/.
=> Attempting to fetch http://freebsd.org/oidn-weights-59bad6bb6344f8fb8205772df3f795c2dc72e23b.tar.gz
fetch: http://freebsd.org/oidn-weights-59bad6bb6344f8fb8205772df3f795c2dc72e23b.tar.gz: Not Found
=> Attempting to fetch http://distcache.FreeBSD.org/ports-distfiles/oidn-weights-59bad6bb6344f8fb8205772df3f795c2dc72e23b.tar.gz
fetch: http://distcache.FreeBSD.org/ports-distfiles/oidn-weights-59bad6bb6344f8fb8205772df3f795c2dc72e23b.tar.gz: Not Found
=> Couldn't fetch it - please try to retrieve this
=> port manually into /usr/ports/distfiles/ and try again.
*** Error code 1

Stop.
make[1]: stopped in /usr/ports/graphics/oidn
*** Error code 1

Stop.
make: stopped in /usr/ports/graphics/oidn
root@dhcp79:/usr/ports/graphics/oidn #


#=#=#=#=#=#=#=#=#=#=#=#=#=#=#=#
And after 'make makesum', 'make ckecksum' succeeds for me.  


root@dhcp79:/usr/ports/graphics/oidn # make makesum
===>  License APACHE20 accepted by the user
===>  License APACHE20 accepted by the user
===>   oidn-1.3.0_1 depends on file: /usr/local/sbin/pkg - found
===>   oidn-1.3.0_1 depends on executable: git - found
===>   oidn-1.3.0_1 depends on executable: git-lfs - found
===> Fetching all distfiles required by oidn-1.3.0_1 for building
root@dhcp79:/usr/ports/graphics/oidn # make checksum
===>  License APACHE20 accepted by the user
===>   oidn-1.3.0_1 depends on file: /usr/local/sbin/pkg - found
===>   oidn-1.3.0_1 depends on executable: git - found
===>   oidn-1.3.0_1 depends on executable: git-lfs - found
===> Fetching all distfiles required by oidn-1.3.0_1 for building
=> SHA256 Checksum OK for oidn-weights-59bad6bb6344f8fb8205772df3f795c2dc72e23b.tar.gz.
=> SHA256 Checksum OK for OpenImageDenoise-oidn-v1.3.0_GH0.tar.gz.
=> SHA256 Checksum OK for OpenImageDenoise-mkl-dnn-eb3e967_GH0.tar.gz.
root@dhcp79:/usr/ports/graphics/oidn #


#=#=#=#=#=#=#=#=#=#=#=#=#=#=#=#
Can you get the same 'oidn-weights-59bad6bb6344f8fb8205772df3f795c2dc72e23b.tar.gz' as the current distinfo was created?  

If not, the distinfo should be updated and distributed in ports tree.  

Cheers,

Mamoru (Japan)
Comment 5 Yuri Victorovich freebsd_committer freebsd_triage 2021-02-25 08:17:33 UTC
(In reply to 1wkmmr from comment #4)

'make distclean fetch checksum' also succeeds for me.
Comment 6 Yuri Victorovich freebsd_committer freebsd_triage 2021-02-25 08:23:01 UTC
It's puzzling, Mamoru, that you are getting this problem.

I think that there is something in your environment that causes it. But I can't think of anything that it could be.

No fallout e-mails arrived for graphics/oidn either. This means that the central package build server can also fetch it without problems.


Yuri
Comment 7 Mamoru IWAKI 2021-02-25 08:49:01 UTC
(In reply to Yuri Victorovich from comment #6)

Hi, Yuri,

Ok, I understand what causes this problem maybe on my side although I have not find it yet :-(

I'll report the cause if I can find it and if it is useful for everyone ;-)

Thank you very much.  

Mamoru
Comment 8 Yuri Victorovich freebsd_committer freebsd_triage 2021-02-25 08:58:36 UTC
(In reply to 1wkmmr from comment #7)

You are welcome!

Please report on your findings.

Best,
Yuri
Comment 9 Vladimir Druzenko freebsd_committer freebsd_triage 2024-02-17 11:16:45 UTC
Close?
Comment 10 Mamoru IWAKI 2024-02-17 15:01:30 UTC
(In reply to Vladimir Druzenko from comment #9)
Although I have not solved this problem yet, nobody other than me reported similar problem until now. So I agree to close this PR.

Thank you so much
Comment 11 Vladimir Druzenko freebsd_committer freebsd_triage 2024-02-17 15:11:03 UTC
Maybe your host is IPv6 only?
Can you fetch distfiles manually using fetch or wget or curl?
Comment 12 Mamoru IWAKI 2024-02-17 15:57:47 UTC
Created attachment 248536 [details]
errornously downloaded file

(In reply to Vladimir Druzenko from comment #11)
Hi, Vladimir

I'm not using IPv6 though I'm not sure the network between my PC and the download site is IPv4 only or not. 

I used fetch, wget2, and curl, then obtained the following same file in any case. 
I attached with this comment. 
It has different size and checksum comparing to distinfo. 

   name = oidn-weights-4322c25e25a05584f65da1a4be5cef40a4b2e90b.tar.gz
   size = 5363

It looks containing some strange text files of which name begin with 'rt*'. 

I have no idea why it happens. 

Please check it out if you have some ideas. 

Thanks
Comment 13 Yuri Victorovich freebsd_committer freebsd_triage 2024-02-17 16:02:34 UTC
This looks puzzling.
Comment 14 Yuri Victorovich freebsd_committer freebsd_triage 2024-02-17 16:25:55 UTC
(In reply to Yuri Victorovich from comment #13)


> I used fetch, wget2, and curl, then obtained the following same file in any case.

What URL are you using with fetch, wget2, curl?
Comment 15 Mamoru IWAKI 2024-02-17 16:26:15 UTC
Comment on attachment 248536 [details]
errornously downloaded file

% curl --output  oidn-weights-4322c25e25a05584f65da1a4be5cef40a4b2e90b.tar.gz http://distcache.FreeBSD.org/ports-distfiles/oidn-weights-4322c25e25a05584f65da1a4be5cef40a4b2e90b.tar.gz
Comment 16 Yuri Victorovich freebsd_committer freebsd_triage 2024-02-17 16:30:33 UTC
(In reply to Mamoru IWAKI from comment #15)

After 'make distclean fetch', what is the size of the file ../../distfiles/oidn-weights-4322c25e25a05584f65da1a4be5cef40a4b2e90b.tar.gz ?

It should be 20605602 bytes.
Comment 17 Mamoru IWAKI 2024-02-17 16:41:53 UTC
(In reply to Yuri Victorovich from comment #16)

I tried that as follows, then the size of the obtained file is 20605734.  
Unfortunately, it is not 20605602 as we expect.

root@:~ # cd /usr/ports/graphics/oidn/
root@:/usr/ports/graphics/oidn # make distclean fetch
===>  Cleaning for git-2.43.2
===>  Cleaning for git-lite-2.43.2
===>  Cleaning for git-tiny-2.43.2
===>  Cleaning for oidn-2.1.0
===>  Deleting distfiles for oidn-2.1.0
===>  License APACHE20 accepted by the user
===>   oidn-2.1.0 depends on file: /usr/local/sbin/pkg - found
===>   oidn-2.1.0 depends on executable: git - found
===>   oidn-2.1.0 depends on executable: git-lfs - found
==> Fetching the oidn/weights subproject with large LFS files
Downloading rt_alb.tza (1.8 MB)
Downloading rt_hdr.tza (1.8 MB)
Downloading rt_hdr_alb.tza (1.8 MB)
Downloading rt_hdr_alb_nrm.tza (1.8 MB)
Downloading rt_hdr_calb_cnrm.tza (1.8 MB)
Downloading rt_ldr.tza (1.8 MB)
Downloading rt_ldr_alb.tza (1.8 MB)
Downloading rt_ldr_alb_nrm.tza (1.8 MB)
Downloading rt_ldr_calb_cnrm.tza (1.8 MB)
Downloading rt_nrm.tza (1.8 MB)
Downloading rtlightmap_dir.tza (1.8 MB)
Downloading rtlightmap_hdr.tza (1.8 MB)
=> OpenImageDenoise-oidn-v2.1.0_GH0.tar.gz doesn't seem to exist in /usr/ports/distfiles/.
=> Attempting to fetch https://codeload.github.com/OpenImageDenoise/oidn/tar.gz/v2.1.0?dummy=/OpenImageDenoise-oidn-v2.1.0_GH0.tar.gz
fetch: https://codeload.github.com/OpenImageDenoise/oidn/tar.gz/v2.1.0?dummy=/OpenImageDenoise-oidn-v2.1.0_GH0.tar.gz: size unknown
fetch: https://codeload.github.com/OpenImageDenoise/oidn/tar.gz/v2.1.0?dummy=/OpenImageDenoise-oidn-v2.1.0_GH0.tar.gz: size of remote file is not known
OpenImageDenoise-oidn-v2.1.0_GH0.tar.gz               3505 kB   19 MBps    00s
=> OpenImageDenoise-mkl-dnn-9bea36e_GH0.tar.gz doesn't seem to exist in /usr/ports/distfiles/.
=> Attempting to fetch https://codeload.github.com/OpenImageDenoise/mkl-dnn/tar.gz/9bea36e?dummy=/OpenImageDenoise-mkl-dnn-9bea36e_GH0.tar.gz
fetch: https://codeload.github.com/OpenImageDenoise/mkl-dnn/tar.gz/9bea36e?dummy=/OpenImageDenoise-mkl-dnn-9bea36e_GH0.tar.gz: size unknown
fetch: https://codeload.github.com/OpenImageDenoise/mkl-dnn/tar.gz/9bea36e?dummy=/OpenImageDenoise-mkl-dnn-9bea36e_GH0.tar.gz: size of remote file is not known
OpenImageDenoise-mkl-dnn-9bea36e_GH0.tar.gz           7022 kB 8093 kBps    01s
=> ROCmSoftwarePlatform-composable_kernel-e85178b_GH0.tar.gz doesn't seem to exist in /usr/ports/distfiles/.
=> Attempting to fetch https://codeload.github.com/ROCmSoftwarePlatform/composable_kernel/tar.gz/e85178b?dummy=/ROCmSoftwarePlatform-composable_kernel-e85178b_GH0.tar.gz
fetch: https://codeload.github.com/ROCmSoftwarePlatform/composable_kernel/tar.gz/e85178b?dummy=/ROCmSoftwarePlatform-composable_kernel-e85178b_GH0.tar.gz: size unknown
fetch: https://codeload.github.com/ROCmSoftwarePlatform/composable_kernel/tar.gz/e85178b?dummy=/ROCmSoftwarePlatform-composable_kernel-e85178b_GH0.tar.gz: size of remote file is not known
ROCmSoftwarePlatform-composable_kernel-e85178b        2102 kB 4485 kBps    01s
=> NVIDIA-cutlass-66d9cdd_GH0.tar.gz doesn't seem to exist in /usr/ports/distfiles/.
=> Attempting to fetch https://codeload.github.com/NVIDIA/cutlass/tar.gz/66d9cdd?dummy=/NVIDIA-cutlass-66d9cdd_GH0.tar.gz
fetch: https://codeload.github.com/NVIDIA/cutlass/tar.gz/66d9cdd?dummy=/NVIDIA-cutlass-66d9cdd_GH0.tar.gz: size unknown
fetch: https://codeload.github.com/NVIDIA/cutlass/tar.gz/66d9cdd?dummy=/NVIDIA-cutlass-66d9cdd_GH0.tar.gz: size of remote file is not known
NVIDIA-cutlass-66d9cdd_GH0.tar.gz                       14 MB   11 MBps    01s
===> Fetching all distfiles required by oidn-2.1.0 for building
root@:/usr/ports/graphics/oidn # ls -l /usr/ports/distfiles/oidn-weights-4322c25e25a05584f65da1a4be5cef40a4b2e90b.tar.gz

-rw-r--r--  1 root  wheel  20605734 Feb 18 01:33 /usr/ports/distfiles/oidn-weights-4322c25e25a05584f65da1a4be5cef40a4b2e90b.tar.gz
root@:/usr/ports/graphics/oidn #
Comment 18 Yuri Victorovich freebsd_committer freebsd_triage 2024-02-17 16:44:21 UTC
(In reply to Mamoru IWAKI from comment #17)

Mamoru,

Could you please e-mail this file to me, if the e-mail would go through with such a large file?


Yuri
Comment 19 Yuri Victorovich freebsd_committer freebsd_triage 2024-02-17 16:57:14 UTC
Mamoru,


Please either e-mail me oidn-weights-4322c25e25a05584f65da1a4be5cef40a4b2e90b.tar.gz, or upload it somewhere and post the URL.

I will look what is different in these files.

The fetch script in the port makes sure that no system differences affect the file, but there could be something that it is missing. I need to figure out what is that.


Yuri
Comment 20 Mamoru IWAKI 2024-02-17 17:01:27 UTC
(In reply to Yuri Victorovich from comment #18)

I uploaded the file in my web page. 
I'll email the URL to you soon.

Thank you for your trouble.
Comment 21 Yuri Victorovich freebsd_committer freebsd_triage 2024-02-17 17:37:19 UTC
(In reply to Mamoru IWAKI from comment #20)

I have it, thank you.

I'll look at the differences when I would have time tonight.

Thank you for sending the followup information.

Yuri
Comment 22 Yuri Victorovich freebsd_committer freebsd_triage 2024-02-18 01:13:34 UTC
(In reply to Mamoru IWAKI from comment #20)

Mamoru,


What does 'make -V TAR' print for you?


Thanks,
Yuri
Comment 23 Mamoru IWAKI 2024-02-18 02:42:56 UTC
(In reply to Yuri Victorovich from comment #22)

Yuri,

I'm sorry, but I accidentally updated my local ports tree. 
So, graphics/oidn in my PC is now 2.2.0.
Anyway, according to the following result, tar command used in my environment is /usr/bin/tar.


root@:~ # cd /usr/ports/graphics/oidn/
root@:/usr/ports/graphics/oidn # make -V TAR
/usr/bin/tar
root@:/usr/ports/graphics/oidn #


In addition, I did the same thing in comments 16 and 17 as follows. 
The result still shows strange file size and sha256.


root@:~ # cd /usr/ports/graphics/oidn/
root@:/usr/ports/graphics/oidn # make distclean fetch
===>  Cleaning for oidn-2.2.0
===>  Deleting distfiles for oidn-2.2.0
===>  License APACHE20 accepted by the user
===>   oidn-2.2.0 depends on file: /usr/local/sbin/pkg - found
===>   oidn-2.2.0 depends on executable: git - found
===>   oidn-2.2.0 depends on executable: git-lfs - found
==> Fetching the oidn/weights subproject with large LFS files
Downloading rt_alb.tza (1.8 MB)
Downloading rt_hdr.tza (1.8 MB)
Downloading rt_hdr_alb.tza (1.8 MB)
Downloading rt_hdr_alb_nrm.tza (1.8 MB)
Downloading rt_hdr_calb_cnrm.tza (1.8 MB)
Downloading rt_ldr.tza (1.8 MB)
Downloading rt_ldr_alb.tza (1.8 MB)
Downloading rt_ldr_alb_nrm.tza (1.8 MB)
Downloading rt_ldr_calb_cnrm.tza (1.8 MB)
Downloading rt_nrm.tza (1.8 MB)
Downloading rtlightmap_dir.tza (1.8 MB)
Downloading rtlightmap_hdr.tza (1.8 MB)
Downloading rt_alb.tza (1.8 MB)
Downloading rt_hdr.tza (1.8 MB)
Downloading rt_hdr_alb.tza (1.8 MB)
Downloading rt_hdr_alb_nrm.tza (1.8 MB)
Downloading rt_hdr_calb_cnrm.tza (1.8 MB)
Downloading rt_ldr.tza (1.8 MB)
Downloading rt_ldr_alb.tza (1.8 MB)
Downloading rt_ldr_alb_nrm.tza (1.8 MB)
Downloading rt_ldr_calb_cnrm.tza (1.8 MB)
Downloading rt_nrm.tza (1.8 MB)
Downloading rtlightmap_dir.tza (1.8 MB)
Downloading rtlightmap_hdr.tza (1.8 MB)
=> OpenImageDenoise-oidn-v2.2.0_GH0.tar.gz doesn't seem to exist in /usr/ports/distfiles/.
=> Attempting to fetch https://codeload.github.com/OpenImageDenoise/oidn/tar.gz/v2.2.0?dummy=/OpenImageDenoise-oidn-v2.2.0_GH0.tar.gz
fetch: https://codeload.github.com/OpenImageDenoise/oidn/tar.gz/v2.2.0?dummy=/OpenImageDenoise-oidn-v2.2.0_GH0.tar.gz: size unknown
fetch: https://codeload.github.com/OpenImageDenoise/oidn/tar.gz/v2.2.0?dummy=/OpenImageDenoise-oidn-v2.2.0_GH0.tar.gz: size of remote file is not known
OpenImageDenoise-oidn-v2.2.0_GH0.tar.gz               3544 kB   10 MBps    00s
=> OpenImageDenoise-mkl-dnn-9bea36e_GH0.tar.gz doesn't seem to exist in /usr/ports/distfiles/.
=> Attempting to fetch https://codeload.github.com/OpenImageDenoise/mkl-dnn/tar.gz/9bea36e?dummy=/OpenImageDenoise-mkl-dnn-9bea36e_GH0.tar.gz
fetch: https://codeload.github.com/OpenImageDenoise/mkl-dnn/tar.gz/9bea36e?dummy=/OpenImageDenoise-mkl-dnn-9bea36e_GH0.tar.gz: size unknown
fetch: https://codeload.github.com/OpenImageDenoise/mkl-dnn/tar.gz/9bea36e?dummy=/OpenImageDenoise-mkl-dnn-9bea36e_GH0.tar.gz: size of remote file is not known
OpenImageDenoise-mkl-dnn-9bea36e_GH0.tar.gz           7022 kB 8180 kBps    01s
=> ROCmSoftwarePlatform-composable_kernel-e85178b_GH0.tar.gz doesn't seem to exist in /usr/ports/distfiles/.
=> Attempting to fetch https://codeload.github.com/ROCmSoftwarePlatform/composable_kernel/tar.gz/e85178b?dummy=/ROCmSoftwarePlatform-composable_kernel-e85178b_GH0.tar.gz
fetch: https://codeload.github.com/ROCmSoftwarePlatform/composable_kernel/tar.gz/e85178b?dummy=/ROCmSoftwarePlatform-composable_kernel-e85178b_GH0.tar.gz: size unknown
fetch: https://codeload.github.com/ROCmSoftwarePlatform/composable_kernel/tar.gz/e85178b?dummy=/ROCmSoftwarePlatform-composable_kernel-e85178b_GH0.tar.gz: size of remote file is not known
ROCmSoftwarePlatform-composable_kernel-e85178b        2102 kB 9393 kBps    01s
=> NVIDIA-cutlass-66d9cdd_GH0.tar.gz doesn't seem to exist in /usr/ports/distfiles/.
=> Attempting to fetch https://codeload.github.com/NVIDIA/cutlass/tar.gz/66d9cdd?dummy=/NVIDIA-cutlass-66d9cdd_GH0.tar.gz
fetch: https://codeload.github.com/NVIDIA/cutlass/tar.gz/66d9cdd?dummy=/NVIDIA-cutlass-66d9cdd_GH0.tar.gz: size unknown
fetch: https://codeload.github.com/NVIDIA/cutlass/tar.gz/66d9cdd?dummy=/NVIDIA-cutlass-66d9cdd_GH0.tar.gz: size of remote file is not known
NVIDIA-cutlass-66d9cdd_GH0.tar.gz                       14 MB   10 MBps    01s
===> Fetching all distfiles required by oidn-2.2.0 for building
root@:/usr/ports/graphics/oidn # ls -l /usr/ports/distfiles/oidn-weights-44ff866123ffd6c26bbc27e5e48e8cd4ec8a1a66.tar.gz
-rw-r--r--  1 root  wheel  20594283 Feb 18 11:38 /usr/ports/distfiles/oidn-weights-44ff866123ffd6c26bbc27e5e48e8cd4ec8a1a66.tar.gz
root@:/usr/ports/graphics/oidn # grep oidn-weights distinfo
SHA256 (oidn-weights-44ff866123ffd6c26bbc27e5e48e8cd4ec8a1a66.tar.gz) = 042707ed810b503c6d6ee0fa0ff1fad48ae04b880f4e7a3ce3efd54ab583e9e7
SIZE (oidn-weights-44ff866123ffd6c26bbc27e5e48e8cd4ec8a1a66.tar.gz) = 20593214
root@:/usr/ports/graphics/oidn #
Comment 24 Yuri Victorovich freebsd_committer freebsd_triage 2024-02-18 05:37:14 UTC
(In reply to Mamoru IWAKI from comment #23)

Mamoru,

The ${TAR} command on your system adds PAX information to tar archives, which isn't supposed to be added in this case.

The tar(1) command supports PAX information in archives with the '--format pax' option, but we explicitly use '--format=bsdtar' in graphics/oidn.

Somehow the tar command works differently on your system, and uses the pax format anyway. Maybe you use altered base system source code by any chance?


Yuri
Comment 25 Yuri Victorovich freebsd_committer freebsd_triage 2024-02-18 06:50:17 UTC
Mamoru,


I am closing this PR because your tar command is a culprit for some reasons local to your system.

There's no bug in the port.

If you would un-gzip the oidn-weights file that is fetched on your system you would find the 'PaxHeader' string in the very beginning. It shouldn't be present in the fetched archives in this port.

There is a mystery why does the http://distcache.FreeBSD.org/ports-distfiles/oidn-weights-4322c25e25a05584f65da1a4be5cef40a4b2e90b.tar.gz contain an invalid file, but there's no fallout which means that this port normally builds successfully.


Best,
Yuri
Comment 26 Mamoru IWAKI 2024-02-18 07:02:05 UTC
(In reply to Yuri Victorovich from comment #25)

Yuri,

Thank you very much for investigation.
I agree with you.
Comment 27 Mamoru IWAKI 2024-02-18 13:43:52 UTC
Created attachment 248559 [details]
change "--format=bsdtar" to "--format=ustar" in Makefile

This patch corrects the problem in this PR. 
The patched Makefile can finish compilation without problem.
Comment 28 Mamoru IWAKI 2024-02-18 15:14:17 UTC
ReOpen: because this is a bug actually.

Note: tar command in FreeBSD-13 has an option --format.
It can be set to ustar out of some possible values, but bsdtar, according to manpages tar(1) and libarchive-formats(5).
However, the original Makefile in this port set bsdtar to this option.
Comment 29 Yuri Victorovich freebsd_committer freebsd_triage 2024-02-18 16:23:00 UTC
(In reply to Mamoru IWAKI from comment #28)

Hi Mamoru,


Did you determine why bsdtar doesn't work on your system, but ustar does?


Thanks,
Yuri
Comment 30 Mamoru IWAKI 2024-02-18 23:18:47 UTC
(In reply to Yuri Victorovich from comment #29)

My system is STABLE-13.

> uname -a
FreeBSD  13.3-STABLE FreeBSD 13.3-STABLE #0 stable/13-n257451-854db1db3f81-dirty: Sun Feb 18 16:34:12 JST 2024     root@:/usr/obj/nanobsd.full/usr/src/amd64.amd64/sys/MYGEN amd64

And, manpages of tar(1) and libarchive-formats(5) do not have any description about bsdtar.
So I think execution of tar with --format=bsdtar is not guaranteed 
although I'm not sure why the execution does not stop with error.

Thanks
Comment 31 Mamoru IWAKI 2024-02-19 00:57:16 UTC
(In reply to Mamoru IWAKI from comment #30)

I checked it on 13.2-RELEASE too. 
bsdtar is not included in the manpages of tar(1) and libarchive-format(5) 

> uname -a
FreeBSD bell1.tmnetwork.local 13.2-RELEASE-p10 FreeBSD 13.2-RELEASE-p10 GENERIC amd64

This system is installed almost everything in normal way.
So I think this PR is not a local problem in my PC, but also in any PCs installed 13.2-RELEASE.
Comment 32 Yuri Victorovich freebsd_committer freebsd_triage 2024-02-19 01:33:45 UTC
Thank you, Mamoru, for your report, and for your persistence.

I updated the port.


Best,
Yuri
Comment 33 commit-hook freebsd_committer freebsd_triage 2024-02-19 01:33:58 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/ports/commit/?id=c22e29d08774bc5ad5eee4406ae59898c5903619

commit c22e29d08774bc5ad5eee4406ae59898c5903619
Author:     Yuri Victorovich <yuri@FreeBSD.org>
AuthorDate: 2024-02-19 01:30:02 +0000
Commit:     Yuri Victorovich <yuri@FreeBSD.org>
CommitDate: 2024-02-19 01:33:41 +0000

    graphics/oidn: Replace format=bsdtar with format=ustar

    ... because the undocumented format bsdtar adds paxheaders on
    some systems, therefore breaking the fetch.

    The documentation for format=bsdtar should be added:
    https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=277139

    PR:             253814
    Reported by:    Mamoru IWAKI <1wkmmr@gmail.com>

 graphics/oidn/Makefile | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Comment 34 commit-hook freebsd_committer freebsd_triage 2024-02-19 01:51:00 UTC
A commit in branch 2024Q1 references this bug:

URL: https://cgit.FreeBSD.org/ports/commit/?id=1a37cea7c744a9bafce50cce3e3b3152aeaaec72

commit 1a37cea7c744a9bafce50cce3e3b3152aeaaec72
Author:     Yuri Victorovich <yuri@FreeBSD.org>
AuthorDate: 2024-02-19 01:30:02 +0000
Commit:     Yuri Victorovich <yuri@FreeBSD.org>
CommitDate: 2024-02-19 01:50:02 +0000

    graphics/oidn: Replace format=bsdtar with format=ustar

    ... because the undocumented format bsdtar adds paxheaders on
    some systems, therefore breaking the fetch.

    The documentation for format=bsdtar should be added:
    https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=277139

    PR:             253814
    Reported by:    Mamoru IWAKI <1wkmmr@gmail.com>

    (cherry picked from commit c22e29d08774bc5ad5eee4406ae59898c5903619)

 graphics/oidn/Makefile | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)