Hi. I am not sure if this is the right place to post this issue. If not, please advice where I can raise awareness for the following issue: The packages directory in the DVD disk image for ARM64 contains wrongfully only AMD64 binaries. E.g. in the directory packages/FreeBSD14:amd64/All in FreeBSD-14.0-RELEASE-arm64-aarch64-dvd1.iso all PKG files are for the AMD64 architecture, while the distribution is ARM64. The name of the directory should be FreeBSD:14:aarch64, imho, and only contain ARM binaries... Thanks!
^Triage: reassign to Release Engineering.
Jessica Clarke noticed that pkg-stage.sh just runs 'pkg config ABI' on the host system to determine the packages to download. This means it always downloads the packages for the running host (both version and architecture), not the target release being built.
Created attachment 249869 [details] pkg_rootdir.patch Untested possible fix: use --rootdir to point pkg at the installed world in dvd/ to determine the ABI of the package set to fetch.
Created attachment 249870 [details] pkg_rootdir.patch Add argument to all pkg invocations so the fetch fetches the right thing.
Another option might be to use -o ABI_FILE=dvd/bin/sh instead of --rootdir. One question is where the database updated by 'pkg update' should really live.
Created attachment 249887 [details] pkg_rootdir.patch So adding --rootdir does have some effects, namely the -o argument to fetch has to be relative to --rootdir, but this version I've tested locally to build a FreeBSD 15 aarch64 release on an FreeBSD 14 host and it did fetch 15.x aarch64 packages and store them in the dvd output directory. I got an error trying to build the actual dvd.iso, but it got past the packages part: cp /usr/home/john/work/freebsd/main/release/scripts/FreeBSD_install_cdrom.conf /usr/obj/usr/home/john/work/freebsd/main/arm64.aarch64/release/dvd/packages/repos/ touch pkg-stage cd dvd && sh /usr/home/john/work/freebsd/main/release/arm64/mkisoimages.sh -b 15_0_CURRENT_aarch64_DVD ../dvd1.iso . Creating `/tmp/efiboot.phxRxv' /tmp/efiboot.phxRxv: 4039 sectors in 4039 FAT12 clusters (512 bytes/cluster) BytesPerSec=512 SecPerClust=1 ResSectors=1 FATs=2 RootDirEnts=512 Sectors=4096 Media=0xf0 FATsecs=12 SecPerTrack=63 Heads=255 HiddenSecs=0 Populating `/tmp/efiboot.phxRxv' Image `/tmp/efiboot.phxRxv' complete makefs: fseeko: No space left on device *** Error code 1 > ll /usr/obj/usr/home/john/work/freebsd/main/arm64.aarch64/release/dvd/packages/ total 10 drwxr-xr-x 4 root wheel 9 Apr 10 16:26 FreeBSD:15:aarch64 lrwxr-xr-x 1 root wheel 18 Apr 10 16:24 freebsd:15:aarch64:64 -> FreeBSD:15:aarch64 drwxr-xr-x 2 root wheel 3 Apr 10 16:26 repos
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=ea2663040d524efd84b12edcbe83884437fdceea commit ea2663040d524efd84b12edcbe83884437fdceea Author: John Baldwin <jhb@FreeBSD.org> AuthorDate: 2024-04-13 21:16:01 +0000 Commit: Colin Percival <cperciva@FreeBSD.org> CommitDate: 2024-04-14 23:45:40 +0000 release: Use the ABI of the target release to fetch packages - Point --rootdir at the installed destdir in the dvd tree. This causes pkg to determine the ABI from the installed destdir instead of the host's binaries. Previously the result was that packages for the host's ABI were always downloaded breaking cross-releases (e.g. arm64 releases built on an amd64 host included amd64 packages on the DVD ISO image rather than arm64 packages). This also handles version mismatches, and I tested this by cross-building a 15.x arm64 release on a 14.x amd64 host. - As a result, pkg now does a chdir(3) to the rootdir before running, so the -o argument to fetch needs to be updated to be relative to rootdir instead of the CWD as make runs. - Add a new ROOTDIR variable to limit references to "dvd" to one place. Ideally ROOTDIR would be an argument to this script so that it didn't really know about the dvd layout at all. - While here, simplify creation of symlinks by just using a longer path to the link name instead of using 'cd' in the shell before invoking ln(1). Also use ln -sf to create the pkg.pkg symlink rather than rm + ln. PR: 278273 Reported by: gatekeeper <tiago.gasiba@gmail.com> Reviewed by: imp, delphij MFC after: 3 days Differential Revision: https://reviews.freebsd.org/D44749 release/scripts/pkg-stage.sh | 23 +++++++++++------------ 1 file changed, 11 insertions(+), 12 deletions(-)
A commit in branch stable/14 references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=70dcf0608cfc767321b16509f615c5dddb9378af commit 70dcf0608cfc767321b16509f615c5dddb9378af Author: John Baldwin <jhb@FreeBSD.org> AuthorDate: 2024-04-13 21:16:01 +0000 Commit: Colin Percival <cperciva@FreeBSD.org> CommitDate: 2024-04-17 05:43:27 +0000 release: Use the ABI of the target release to fetch packages - Point --rootdir at the installed destdir in the dvd tree. This causes pkg to determine the ABI from the installed destdir instead of the host's binaries. Previously the result was that packages for the host's ABI were always downloaded breaking cross-releases (e.g. arm64 releases built on an amd64 host included amd64 packages on the DVD ISO image rather than arm64 packages). This also handles version mismatches, and I tested this by cross-building a 15.x arm64 release on a 14.x amd64 host. - As a result, pkg now does a chdir(3) to the rootdir before running, so the -o argument to fetch needs to be updated to be relative to rootdir instead of the CWD as make runs. - Add a new ROOTDIR variable to limit references to "dvd" to one place. Ideally ROOTDIR would be an argument to this script so that it didn't really know about the dvd layout at all. - While here, simplify creation of symlinks by just using a longer path to the link name instead of using 'cd' in the shell before invoking ln(1). Also use ln -sf to create the pkg.pkg symlink rather than rm + ln. PR: 278273 Reported by: gatekeeper <tiago.gasiba@gmail.com> Reviewed by: imp, delphij MFC after: 3 days Differential Revision: https://reviews.freebsd.org/D44749 (cherry picked from commit ea2663040d524efd84b12edcbe83884437fdceea) release/scripts/pkg-stage.sh | 23 +++++++++++------------ 1 file changed, 11 insertions(+), 12 deletions(-)
Hi. First of all, thanks for the awesome work you put into fixing the problem that I reported. Now, sorry if this is a dumb question... :-) I see that there is a commit to the stable/14 branch that references this bug... Does it mean that this bug report can/should be closed? Should I do it? Cheers.
I'm planning on merging this to stable/13 next week, at which point I'll close this PR (unless I forget, which is always possible).
A commit in branch stable/13 references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=835b1bd4045a7ef56f58868fc150d099b2ce135b commit 835b1bd4045a7ef56f58868fc150d099b2ce135b Author: John Baldwin <jhb@FreeBSD.org> AuthorDate: 2024-04-13 21:16:01 +0000 Commit: Colin Percival <cperciva@FreeBSD.org> CommitDate: 2024-04-24 05:04:44 +0000 release: Use the ABI of the target release to fetch packages - Point --rootdir at the installed destdir in the dvd tree. This causes pkg to determine the ABI from the installed destdir instead of the host's binaries. Previously the result was that packages for the host's ABI were always downloaded breaking cross-releases (e.g. arm64 releases built on an amd64 host included amd64 packages on the DVD ISO image rather than arm64 packages). This also handles version mismatches, and I tested this by cross-building a 15.x arm64 release on a 14.x amd64 host. - As a result, pkg now does a chdir(3) to the rootdir before running, so the -o argument to fetch needs to be updated to be relative to rootdir instead of the CWD as make runs. - Add a new ROOTDIR variable to limit references to "dvd" to one place. Ideally ROOTDIR would be an argument to this script so that it didn't really know about the dvd layout at all. - While here, simplify creation of symlinks by just using a longer path to the link name instead of using 'cd' in the shell before invoking ln(1). Also use ln -sf to create the pkg.pkg symlink rather than rm + ln. PR: 278273 Reported by: gatekeeper <tiago.gasiba@gmail.com> Reviewed by: imp, delphij MFC after: 3 days Differential Revision: https://reviews.freebsd.org/D44749 (cherry picked from commit ea2663040d524efd84b12edcbe83884437fdceea) release/scripts/pkg-stage.sh | 23 +++++++++++------------ 1 file changed, 11 insertions(+), 12 deletions(-)
Should be fixed now on all supported branches. Please take a look at 14.1-BETA1 and 13.4-BETA1 when those happen to make sure everything is working properly. :-)