Bug 270506 - Unable to reach loader menu with 13.2-RCx ISOs on amd64
Summary: Unable to reach loader menu with 13.2-RCx ISOs on amd64
Status: Open
Alias: None
Product: Base System
Classification: Unclassified
Component: misc (show other bugs)
Version: 13.2-RELEASE
Hardware: amd64 Any
: --- Affects Only Me
Assignee: freebsd-bugs (Nobody)
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2023-03-28 20:28 UTC by Michael Reim
Modified: 2023-04-10 23:12 UTC (History)
6 users (show)

See Also:


Attachments
Example of the EFI output (889 bytes, text/plain)
2023-03-28 20:28 UTC, Michael Reim
no flags Details
iodd dmesg output (560 bytes, text/plain)
2023-03-30 04:26 UTC, Michael Reim
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Michael Reim 2023-03-28 20:28:09 UTC
Created attachment 241166 [details]
Example of the EFI output

I've noticed that I'm unable to boot FreeBSD when using the latest RC ISO images for 13.2 on amd64 (i.e. FreeBSD-13.2-RC5-amd64-disc1.iso). The problem is fully reproducible on multiple machines both with BIOS and EFI. On the former I end up only getting a blinking cursor instead of the expected beastie menu, on EFI systems I get more information (see attached file).

The ISO image for i386 is not affected and neither is the amd64 memstick version. I have tried out -RC4, too, and it shows the same behavior. Also got -RC3 and -RC2 from a mirror that was not up to date and they are affected as well. The only older ISO that I still had is -BETA1 - and that one boots just fine.

I've posted about this on Reddit, asking if anybody kept older ISOs and some people have tried to reproduce my problem without success. The important information here may be that I'm using a USB CD emulator (the iodd-2531 to be precise).

I cloned the source and checked out the various commits of releng/13.2 when new betas or RCs happened, built the system and ran "make cdrom" to test the ones that I couldn't get from the net. After doing so, my conclusion was that something broke between -BETA1 and -BETA2. This conclusion was wrong, though: For good measure I built the last known good one myself, too, and ended up with an ISO showing the broken behavior! I'm really not sure what is happening here.

A little more context: I ran into a similar problem before when testing a snapshot of MidnightBSD 3.0 a while ago. After I reported the problem, a new ISO was rolled and I was asked to try that. It worked just fine. So yesterday I asked Lucas Holt of MidnightBSD what he had done back then. It seems he found that something had been truncated even though the machine had plenty of space available. Eventually he just ran "make clean" in /usr/src/release before building a new ISO and that fixed it in his case. So there was no code change involved.

I just tried out running "make clean" and then "make cdrom" again with the already compiled -BETA1 code. Unfortunately the resulting ISO still is corrupt. Then I tried out the latest -CURRENT snapshot and found that also unable to get to the loader menu...

I'm currently almost out of ideas on how to narrow this issue down. If anybody has any suggestions, I'll be happy to try them out. Also always willing to provide more info if that could be useful (since the problem happens on all machines that I tested the ISOs with I did not attach a dmesg or output of dmidecode, but I of course could).

Unless somebody proposes another strategy, I'll wipe the test machine and try building -BETA1 again cleanly and without using ccache to see if I can get a working ISO built.
Comment 1 Warner Losh freebsd_committer freebsd_triage 2023-03-29 12:33:30 UTC
Beta1 to rc2 is a big range.
Can you reproduce this in qemu?
I'm traveling so responses may be delayed. This one has my head hurting...
Comment 2 Jessica Clarke freebsd_committer freebsd_triage 2023-03-29 17:25:44 UTC
Are you using release/release.sh to perform the release build inside a clean chroot built from the same sources, or doing it with whatever versions of host tools you have on your system?
Comment 3 Ed Maste freebsd_committer freebsd_triage 2023-03-29 20:39:55 UTC
One suggestion to try, since you have a BETA1 that works (the FreeBSD-built one) and one that should be ~identical, but fails (the one you built yourself): install py37-diffoscope, and run it on the two ISOs

$ diffoscope bad.iso good.iso

with any luck they'll be mostly the same, and diffoscope will have a small set of differences to report. We know makefs cd9660 is not 100% reproducible (see e.g. PR270435) but the differences should be minimal.
Comment 4 Michael Reim 2023-03-30 04:26:11 UTC
Created attachment 241199 [details]
iodd dmesg output
Comment 5 Michael Reim 2023-03-30 04:30:47 UTC
(In reply to Warner Losh from comment #1)

FreeBSD-13.2-RC5-amd64-disc1.iso which won't work with my iodd works perfectly fine both with qemu and virtualbox. Whatever is the issue here seems to be somewhat subtle and is probably only exposed by the way the hardware emulates a CD-ROM drive.

I've added the part of my dmesg with the output that shows when I attach the device on my workstation just to give everybody an idea of what that thing accually is. Please ignore the "Medium not present" line; I think I didn't have an ISO selected in the iodd's menu at that point.
Comment 6 Michael Reim 2023-03-30 04:35:28 UTC
(In reply to Jessica Clarke from comment #2)

Good point! No, I haven't. I cloned the source and checked out the various commits manually. I admit not having used release.sh in a while - since before the git transition. Will do some reading and then try again if I can get an ISO built that mostly matches the official -BETA1 image that I have. Thanks!
Comment 7 Michael Reim 2023-03-30 04:42:46 UTC
(In reply to Ed Maste from comment #3)

Thanks for the suggestion! I didn't know that tool but it looks very helpful. Right now the self-rolled ISO is way different from the official one (filesize of 1029120000 bytes vs. 1076826112 that it should have). As pointed out by Jessica I should have followed the official procedure more closely. Will attempt a diff again once that is done.

Fortunately after trying Warner's suggestion it looks like the issue is limited to my particular virtual CD-ROM device (it's closed-source and who knows what its firmware does?) and thus likely doesn't affect many people. It's probably still worth investigating, though, as FreeBSD ISOs used to work on that hardware, too.
Comment 8 Warner Losh freebsd_committer freebsd_triage 2023-03-30 15:48:07 UTC
Yea, if we can reproduce this, then I'm very keen on fixing it before the release. If we can't, I'm content to let Ed's preening of makefs proceed so that it's more likely to work next time.  I've not seen such pickiness in ISO images since I was playing around with my ArcStation I windows NT MIPS box back in the 90s which would only boot certain ISOs I could create for it, but not others... It was so random I never could get to the bottom of it...
Comment 9 Henrich Hartzer 2023-04-10 17:39:29 UTC
I tested this on QEMU and was able to boot the 13.2-RELEASE ISO without issue. I will try this on some real hardware and report back.
Comment 10 Henrich Hartzer 2023-04-10 23:12:16 UTC
I apologize, I meant VirtualBox, not Qemu. I did test both EFI and BIOS for VirtualBox, and both were able to install.

I was able to test a burned DVD image of -RELEASE with two machines and had no issues (at least with getting to the curses prompt -- I did not try to install). One more had an issue, but it may have been the drive, and I did not verify with an old CD that it was not the drive. So far so good on this end.