In vesa.ko there is a function that gets called when resuming from sleep, e.g. at resuming after S3 suspend via 'zzz'. This function does a BIOS call, which is related to restoring the graphics cards' previous state it had before powering off. On Nvidia cards this BIOS function seems to be implemented in a different way than on most other cards. For this reason, calling this BIOS function causes the Nvidia graphics driver to hang, failing to resume. (For technical background, read my discussion with jkim in PR 224069: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=224069 ) Reproducing the bug is easy: -Install FreeBSD (eg using GENERIC kernel). -Enable sc (kern.vty="sc" in /boot/loader.conf). -Install xorg. Install and configure nvidia driver. Reboot and start xorg via startx. -Enter "zzz" in a xterm. -Watch the system/driver hang, keyboard (PS/2) and mouse becoming unresponsive when attempting to switch back to graphics mode. -Hitting the power button results no visible change, until after timeout a message about an unresponsive stop job (presumably the nvidia driver) appears shortly before powering off. For reproducing the bug it is essential to use GENERIC kernel! Because, working around by building and installing a custom kernel without "options VESA" fixes the hang after suspend/resume. It is also important to not have vt and its helper modules (vt_efifb etc) in the kernel, as these pull in the vesa.ko showstopper module. Already back in 2017 I found that skipping (commenting out) the Nvidia BIOS call fixes the issue, making resume work reliably. So I believe the proper fix would be: 1. check whether the graphics card is Nvidia 2. if it is Nvidia, skip that BIOS call in /usr/src/sys/dev/fb/vesa.c line 520. Pseudocode for a patch might look like this: (+)if (! nvidia_card_is_installed) { x86bios_intr(®s, 0x10); (+)} Side note: All my systems use the sc console, not the vt console. So I do not know the system behaviour when using vt. Thus please use sc in text mode when reproducing the bug!
Created attachment 223218 [details] Disable POST and save/restore states on NVIDIA cards Please try the attached patch. I cannot test it because I don't use syscons(4) any more.
Thank you very much, Jung-uk! I am going to test the patch on my computers using different nvidia cards/drivers. To make sure it works reliably, I want to test long enough, to accumulate sufficient uptime and suspend/resume cycles. So it might take a few days until I report back.
Sorry for late update. I was busy with other things. Tested the patch. It does not work. Still hangs in text mode when resuming. Not sure for which exact cause yet. Apparently I didn't express myself clearly enough; it is *only* the LOAD_STATE call which breaks resume and needs to be omitted in case of Nvidia card/chip. I verified this is still valid by commenting out the x86bios_intr() call in the case STATE_LOAD: of vesa_bios_save_restore(). So I believe the other VESA calls, including POST, do *not* have a negative impact on suspend/resume. I didn't test yet whether vesa_find_pci_device() actually finds the card which responds to the VESA BIOS call (but will do soon using some debug printfs). So I can not rule out yet that a problem there could be the potential cause for the patch not working. Another issue I am not yet clear of whether it matters: There are some OEMs who had in some cases their onboard video BIOS at other locations than C000. I remember some cases I personally encountered, where video BIOS was at E000. For this reason I am not really sure whether the approach of checking for a C000 BIOS start address is 100% safe. I am now thinking about scanning the OEM string which gets returned by function 4F00 for "nvidia" (case-independent), eg the string the VESA 1.2 OEMStringPtr points to. This approach would be independent of the Option BIOS memory address. I have about ten different Nvidia cards and onboard chips, NV4 and higher, and will read out their OEMString via debug printfs, to find out whether this alternative approach could be viable. As I am currently moving my hardware lab, it will take about 1-3 weeks until I report back, maybe with an updated patch.