Bug 290295 - aws ec2 freebsd 14 arm serial console gone
Summary: aws ec2 freebsd 14 arm serial console gone
Status: New
Alias: None
Product: Base System
Classification: Unclassified
Component: arm (show other bugs)
Version: 14.3-STABLE
Hardware: arm64 Any
: --- Affects Many People
Assignee: freebsd-arm (Nobody)
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2025-10-16 14:02 UTC by clive
Modified: 2025-11-12 15:41 UTC (History)
6 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description clive 2025-10-16 14:02:31 UTC
I'm not sure what changed in aws around Sep or Oct 2025, I noticed that serial console is gone on arm64 instances. As to amd64 instances, all works. I ran more tests on various snapshots and found that in FreeBSD 15, the issue is resolved after 5.0-PRERELEASE-arm64-20250822. Then I tested several kernel builds by reverting commits one by one, and luckily found the one-line fix 

https://cgit.freebsd.org/src/commit/?h=releng/15.0&id=a685a263b803012d9546abaca4dcf52f80c6b8d6

    Although it's a one line patch, but for me it's not easy to apply it on stable/14. Either find the equivalent bits in stable/14 and fix it, or apply all the fundamental changes in stable/15 (uart, acpica, maybe pci as well) to fix it.

    Since 14.4 release is still months away, lwhsu@ recommended opening a ticket here. I’m happy to test any potential fix.

    Additional tests I did before submitting the ticket:

Up to table/14-n272680-258cde4b2e48 https://cgit.freebsd.org/src/commit/?h=stable/14&id=258cde4b2e4832bfaa49c800754aec8800a388c7
The serial console is still missing.

Up to stable/15-n280713-24f4ec797e9b https://cgit.freebsd.org/src/commit/?h=stable/15&id=24f4ec797e9b19abd4488554c78e7ecc27313428
The serial console is still detected and and usable.

    Both are GENERIC kernel. Tested on instance types: t4g.small and m8g.2xlarge. 

    Sorry for the long CC list. It's based on the commit log, reviews, and of course our aws hero.
Comment 1 Warner Losh freebsd_committer freebsd_triage 2025-10-16 16:28:15 UTC
The fix is one line, but it's a fix to the changes to parse these tables, which apparently haven't been merged.
Comment 2 Colin Percival freebsd_committer freebsd_triage 2025-10-16 17:02:05 UTC
Thanks, I'm buried in too much 15.0 stuff right now but I'll make sure this is fixed in 14.4.
Comment 3 dwmw2 2025-11-12 11:49:33 UTC
Argh. Apparently the question I should have asked before permitting that change is not "have you checked FreeBSD?" but "have you actually *booted* a *release* version of FreeBSD and observed that it works?".

It looks like we merely checked the FreeBSD source code and saw that it would already work in mainline, and didn't spot that it was a recent change not in any existing release.

I will ensure that more care is taken in future; apologies for that.

Reverting this change now that it's been out there for so long is non-trivial (feel free to ping me early if such things happen in future!). If there's any way we could get the corresponding FreeBSD changes into the available AMIs as soon as possible, that would be much appreciated, Colin?
Comment 4 Colin Percival freebsd_committer freebsd_triage 2025-11-12 14:54:29 UTC
Hi David,

I'm not sure what you mean by "into the available AMIs" -- by definition, AMIs are immutable.  We could potentially roll out an Errata Notice for past releases for this, but it would only help after an instance is updated and reboots (as it does when it first boots, but you'd still lose the console output from the first time it boots).

We don't have any mechanism in place for re-rolling releases yet -- that's a to-do for 2026, and will probably only apply to 15.0 and later since pkgbase makes it much easier to do that.
Comment 5 dwmw2 2025-11-12 15:36:43 UTC
Understood, thanks. I wasn't sure how often, if ever, you were rolling out new versions of e.g. the 14.3 AMIs. So any fix for this on the guest side is 14.4 or later, and is some months off, and doesn't resolve things for existing instances anyway. I shall take that information, and my baseball bat and most disapproving frown, and go and attempt to do something useful with the combination thereof...
Comment 6 Colin Percival freebsd_committer freebsd_triage 2025-11-12 15:41:24 UTC
If there are specific customers this is affecting, I could manually re-roll 14.3 for them (especially if we do an EN with this fix).  But it wouldn't be an "Official FreeBSD image" and I wouldn't want to publicize such images widely because I don't want to teach people to trust non-official images.
Comment 7 Colin Percival freebsd_committer freebsd_triage 2025-11-12 15:41:57 UTC
(And yes, 14.4 is scheduled for March 10th.)