Bug 258264 - FreeBSD Handbook guidance for NVIDIA graphics may lead to FreeBSD not booting/starting
Summary: FreeBSD Handbook guidance for NVIDIA graphics may lead to FreeBSD not booting...
Status: Closed Overcome By Events
Alias: None
Product: Documentation
Classification: Unclassified
Component: Books & Articles (show other bugs)
Version: Latest
Hardware: amd64 Any
: --- Affects Some People
Assignee: freebsd-doc (Nobody)
URL:
Keywords:
Depends on:
Blocks: handbook-2022
  Show dependency treegraph
 
Reported: 2021-09-04 11:00 UTC by otto.e.petaja
Modified: 2023-02-20 08:13 UTC (History)
9 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description otto.e.petaja 2021-09-04 11:00:49 UTC
So I have tried installing FreeBSD many times by now, which I was able to do finally by looking at resources outside of the handbook, but the handbook itself would not get me a functional system.The problem is with section 5.8.1 about installing nvidia drivers.

The issue is that adding "nvidia_load="YES"" into /boot/loader.conf as the handbook says would make my system get stuck while booting. Generally the bootloader goes as far as saying that it's loading the nvidia driver and says the EFI frame buffer information or whatever, but it stops moving forward completely after that. 

Instead, I had to add kld_list="nvidia-modeset nvidia" into /etc/rc.conf and that allowed the system to properly load the driver. 

As far as the scale goes, I have seen the forum posts that allowed me to get the system working, so some others have had the issue as well. However, I have also seen some people report not having the problem, so it seems to not be affecting everyone either.
Comment 1 Danilo G. Baio freebsd_committer freebsd_triage 2022-01-08 12:07:10 UTC
Adding danfe@ and x11@.

debdrup@ said that maybe nvidia-modeset is the preferred module now.

Could you help here? We want to know the proper and recommended way to load nvidia or nvidia-modeset is?

So we can combine the same information in ports and handbook.

Thank you in advance.


https://cgit.freebsd.org/ports/tree/x11/nvidia-driver/files/pkg-message.in
https://docs.freebsd.org/en/books/handbook/x11/
Comment 2 Alexey Dokuchaev freebsd_committer freebsd_triage 2022-01-08 13:35:48 UTC
(In reply to Danilo G. Baio from comment #1)
> debdrup@ said that maybe nvidia-modeset is the preferred module now.
> Could you help here?
It's all thoroughly documented in the x11/nvidia-driver/files/pkg-message.in, I don't quite see what is the problem here.
Comment 3 Danilo G. Baio freebsd_committer freebsd_triage 2022-01-08 13:53:02 UTC
(In reply to Alexey Dokuchaev from comment #2)

That helps, thanks.

So we need to adjust the handbook following the port/pkg-message or point users to there.
Comment 4 Graham Perrin freebsd_committer freebsd_triage 2022-01-09 05:11:23 UTC
(In reply to Danilo G. Baio from comment #3)

A few thoughts. 

From <https://docs.freebsd.org/en/books/porters-handbook/pkg-files/index.html#porting-message>: 

> pkg-message must contain only information that is vital to setup and 
> operation on FreeBSD, and that is unique to the port in question.

<https://www.freshports.org/x11/nvidia-driver/#message> there's a paragraph across eight lines about linux_enable="YES", which is not unique to any driver for NVIDIA. 

Instead, consider a more concise reference to: 
<https://docs.freebsd.org/en/books/handbook/linuxemu/#linuxemu-lbc-install>


The installed README (from NVIDIA) is thorough, however more than sixty-two thousand words on a single page is probably a turn-off for a significant proportion of users. 

Moreover, it's better to have sight of a README prior to installation. 

Paginated HTML e.g. (for the currently ported version) <https://http.download.nvidia.com/XFree86/FreeBSD-x86_64/470.86/README/> is more user-friendly. 

The FreeBSD Handbook might usefully refer to <https://www.nvidia.com/en-us/drivers/unix/> with hints: 

* to scroll down to the FreeBSD x64 section to find what's 
  closest to the currently ported version

* to then find the README under ADDITIONAL INFORMATION; and 
  to _not_ download the driver from NVIDIA's site if the intention is 
  to install the port

  – NVIDIA drivers in the FreeBSD ports collection benefit from 
    specialist maintainance.


(Words to that effect.)

Thanks
Comment 5 Austin Shafer 2022-01-11 00:18:37 UTC
(In reply to otto.e.petaja from comment #0)

As the port message says the nvidia module should be loaded from rc.conf, not from loader.conf. Using loader.conf has a high chance of crashing. The problem iirc is that for whatever reason at loader time we can't grow the reserved memory for the kernel (efi reserved memory or something like that) and that's when you see it hang.

We should definitely update the handbook if it still recommends loader.conf.
Comment 6 Gary Jennejohn 2022-01-11 10:03:16 UTC
(In reply to Austin Shafer from comment #5)
I don't boot using EFI but rather have my BIOS in legacy mode.  I load nvidia-driver from loader.conf and have never seen a hang in the many years I've been doing that.  So the handbook could reflect that EFI booting may cause a problem using loader.conf, but that booting in legacy mode probably will not.
Comment 7 Pau Amma 2022-01-11 16:29:34 UTC
(In reply to Gary Jennejohn from comment #6)

It could, but I don't think it should unless legacy boot + loading nvidia-modeset from rc can cause problems. (Can it?)

In the same way that the ethernet+wifi aggregation section (https://docs.freebsd.org/en/books/handbook/advanced-networking/#network-aggregation example 3) was changed (in https://cgit.freebsd.org/doc/commit/en_US.ISO8859-1/books/handbook/advanced-networking/chapter.xml?id=30c4d8ce1add5bfb355434db5924fbf77e21d8d4) from something that failed in some cases but not all to something that was known to work in all cases even if some could have worked without the change (see the "In theory" note), it would be better to recommend a method with no reported problems, for simplicity.
Comment 8 Chris Hutchinson 2022-01-11 17:04:55 UTC
(In reply to Austin Shafer from comment #5)
The loader.conf(5) vs. rc.conf(5) is really kind of a
red herring in that there were a couple of (FreeBSD)
versions that initially struggled with memory mapping
with (U)EFI that *might* cause a problem. But those
(versions) have since passed. I have had zero problems
with 7 different (nvidia) versions in 7 different
boxes / laptops in *either* (U)EFI or BIOS boot using
loader.conf(5). If one strongly feels compelled to write
something regarding loader.conf(5), IMHO it should read
MAY pose a problem. I've been using loader.conf exclusively
for many years (since 4.2) w/o any problems FWIW. :-)
Comment 9 Graham Perrin freebsd_committer freebsd_triage 2022-01-21 02:29:40 UTC
(In reply to Alexey Dokuchaev from comment #2)

> It's all thoroughly documented in the 
> x11/nvidia-driver/files/pkg-message.in, I don't quite see 
> what is the problem here.

I shared a screenshot (with highlights) of a FreshPorts view of the package message. I'm accused of posting:

    misinformation

I don't quite see what is the problem here: 

    <https://forums.freebsd.org/post/551939>

If someone can explain, technically, my mistake, I'll be grateful. Thanks.
Comment 10 Graham Perrin freebsd_committer freebsd_triage 2022-01-21 02:30:39 UTC
(In reply to Graham Perrin from comment #9)

Sorry, correcting the URL: <https://forums.freebsd.org/posts/551939>
Comment 11 Pau Amma 2022-01-21 08:01:47 UTC
(In reply to Graham Perrin from comment #10)
I don't have an answer to that question, but going on a tangent: next time could you post text copies (with either inline text annotations or comments below) instead of screenshots with added highlights? Those are completely unusable for someone using a screenreader. (https://webaim.org/articles/visual/blind#screenreaders for context.)
Comment 12 Stefan B. 2022-03-14 06:23:14 UTC
Graham Perrin told me here https://forums.freebsd.org/threads/why-its-so-complicated.84408/post-560016 to add information.

In general, for all drivers >340 the correct way is to load nvidia-modeset in /boot/loader.conf. (Source is the Nvidia README, confirmed by my experience.)
Loading it only later (via /etc/rc.conf) can cause several issues, like xrandr/EDID not working correctly, problems with suspend/resume etc.

However there are issues with the UEFI boot code which reportedly cause hangs due to insufficient free memory, which causes problems when nvidia.ko is being pulled in by nvidia-modeset. I don't know about details, as I avoid using UEFI whenever possible.

So it depends on whether and when these UEFI loader issues are solved in the particular FreeBSD version, whether nvidia-modeset can be loaded via /boot/loader.conf (optimum) or has to be loaded via /etc/rc.conf (suboptimal work-around).

The handbook nvidia section is of no use, as it is completely out of date and no help at all for FreeBSD newcomers. It even still mentions driver-304, which has been defunct since 2017 due Xorg ABI changes.
Comment 13 Graham Perrin freebsd_committer freebsd_triage 2022-07-16 11:13:04 UTC
(In reply to PauAmma from comment #11)

Green highlights drawn across two phrases: 


> kldload nvidia

> sysrc kld_list+=nvidia

pink: 


> hangs when starting X11 server

yellow: 


> or observe

pink across two phrases: 


> Validated MetaModes:

> NULL

yellow: 


> replace ``nvidia'' with ``nvidia-modeset''
Comment 14 Graham Perrin freebsd_committer freebsd_triage 2022-07-16 11:21:39 UTC
(In reply to Graham Perrin from comment #13)

Reading the package message in isolation (without the context of this bug report, without the abuse elsewhere), I see: 

1. nvidia first, the primary recommendation

2. nvidia-modeset as an alternative _only if_ either 

   a) the NULL message is observed; or 

   b) starting X11 server results in a hang.
Comment 15 Sean Farley freebsd_committer freebsd_triage 2022-07-16 16:27:01 UTC
From a discussion I had awhile back after the MFC of https://cgit.freebsd.org/src/commit/?id=4d6047edb675e52b8fad57135ab3ded8e66d0dac:

> I have run into an issue with this change on my workstation running
> 12-STABLE.  It prevents the boot from going further than posting the
> "EFI framebuffer information" section if I load the nvidia driver and
> cpu_microcode (intel-ucode.bin) firmware.  Removing either from
> loader.conf gets it past that point.
>
> If the Intel firmware loads after nvidia, I see an additional message
> printed twice:
>
>      efi_check_space: Unable to expand staging area
>
> I can work around this by loading either of those drivers from
> /etc/rc.conf, but is this something of concern?  For my workstation, I
> prefer loading both from the loader.  It is best to have the microcode
> update loaded as early as possible.  It is also nice to have the nvidia
> driver to panic early, if it does, so graid does not have to rebuild the
> volume.

It becomes an either/or between VMware and Nvidia.  For myself, I rebuild the kernel anyway, so I just reverted it locally.  I have tested against the 13.0 loader.efi without success.  It has been awhile since I checked if there is newer code that fixes it.

Obviously, this method is not something that should be in the handbook yet could help people who come across this issue.
Comment 16 Graham Perrin freebsd_committer freebsd_triage 2022-12-19 04:10:40 UTC
<https://wiki.freebsd.org/Graphics> now includes NVIDIA graphics. Basics (including a cross-reference to this bug report).
Comment 17 Sergio Carlavilla Delgado freebsd_committer freebsd_triage 2023-02-16 07:07:47 UTC
Fixed in https://cgit.freebsd.org/doc/commit/?id=4d1de1c7246d147e1c428b542350695b8fedf84e