Created attachment 145720 [details] x11/nvidia-driver diff This is a very simple slave port of nvidia-driver. Its purpose is simple: for use with Optimus, one needs access to the nVIDIA kmod and the nVIDIA libGL.so.1 and libglx.so. nvidia-driver automatically creates symlinks of the nVIDIA versions of those into the system library directories, which interferes with Optimus by rendering the Intel card unusable. This port installs them into .nvidia_optimus/ instead of .nvidia/ and will therefore not symlink them. There are very small changes to x11/nvidia-driver as well, which I've also attached.
Created attachment 145721 [details] This is the SHAR archive of the port.
I normally ask for test logs as proof of QA for new ports, but since this is a simple slave port, I'll give you the benefit of the doubt. Moving to patch-ready
I'd rather not create another slave port, but add an OPTION to the master port instead. Would it be fine with you? Slaves (we already have a few of nvidia-driver) add certain non-zero maintenance burden, and in the long run many of them we have right now in the Ports Tree will likely go away once the framework learns sub-packages. I'll post a patch for reviews shortly.
(In reply to Alexey Dokuchaev from comment #3) > I'd rather not create another slave port, but add an OPTION to the master > port instead. Would it be fine with you? > > Slaves (we already have a few of nvidia-driver) add certain non-zero > maintenance burden, and in the long run many of them we have right now in > the Ports Tree will likely go away once the framework learns sub-packages. > > I'll post a patch for reviews shortly. An option would work fine, but it might be pertinent to keep a very simple slave port that simply sets the default options to include an 'Optimus' option as well. This allows its installation via `pkg install'.
(In reply to Alexey Dokuchaev from comment #3) > I'd rather not create another slave port, but add an OPTION to the master > port instead. Would it be fine with you? > > Slaves (we already have a few of nvidia-driver) add certain non-zero > maintenance burden, and in the long run many of them we have right now in > the Ports Tree will likely go away once the framework learns sub-packages. > > I'll post a patch for reviews shortly. Hiya, Have you made any progress with a patch yet? Thanks, David
(In reply to Alexey Dokuchaev from comment #3) > I'd rather not create another slave port, but add an OPTION to the master > port instead. Would it be fine with you? > > Slaves (we already have a few of nvidia-driver) add certain non-zero > maintenance burden, and in the long run many of them we have right now in > the Ports Tree will likely go away once the framework learns sub-packages. > > I'll post a patch for reviews shortly. Hiya, As you have not replied in a month I went ahead and redone this port today in the manner of an OPTION. Newly attached are the updated patch to x11/nvidia-driver and an (extremely basic) slave port that simply manipulates default options, so that nvidia-driver-optimus will be installable via `pkg`.
Created attachment 147752 [details] Patch to x11/nvidia-driver adding OPTIMUS option.
Created attachment 147753 [details] Simple slave port to x11/nvidia-driver which sets as a default option the OPTIMUS option.
Assigned to nvidia-driver maintainer.
Thanks for going the OPTIONS knob way! Before proceeding with the patch, I'd like to update nvidia-driver ports (stable and -304) to their recent versions. Would you mind testing this patch and report if it works fine for you (both with and without OPTIMUS option)? http://people.freebsd.org/~danfe/nvidia-update.diff
Hi, I've manually merged your patch into the already-patched x11/nvidia-driver port which I had attached to this PR. Linked is the diff between the original attached x11/nvidia-driver/Makefile and the new one after applying your patch. https://dpaste.de/M8Ym This works fine both Optimus or not.
Thanks for testing. I've committed the update as ports 372198; Optimus support will follow shortly. Since new version 340.58 was released today, I will probably commit Optimus support along with an update, so you might want to test 340.58 as well.
Created attachment 153729 [details] Patch to x11/nvidia-driver adding OPTIMUS option. As part of my final attempt to get this committed I have updated the patch to work against nvidia-driver in ports head.
Just a observation, is it really needed that the libGL.so and libEGL.so get installed in a different location? For one this breaks the graphics/libGL and graphics/libEGL scripts that expect the lib/.nvidia/ syntax. Also nvidia only installed either non-optimus or the optimus variant so would it matter?
(In reply to Koop Mast from comment #14) More than needed, this is the primary purpose of this patch: the installation to an alternate path and consequent prevention of the libGL and libEGL scripts from behaving as though the nvidia-driver is installed. On nVIDIA Optimus, the main driver used is Intel, and consequently, Intel's libGL.so.1 and libEGL.so.1. The nVIDIA driver is used only in conjunction with VirtualGL and a special slave X server (which has no display of its own), which uses LD_PRELOAD manipulation to use a special wrapper libGL.so that calls into both nVIDIA's libGL and Intel's libGL.
(In reply to David Mackay from comment #15) Ok that is clear. I don't know what danfe@ will say but could you add a some comments so this is clear when looking at the Makefile? The nvidia-driver Makefile is complex enough as it is :)
Hi there, has anyone tested this recently? If so, what's the result?
It seems that the port is still not committed to the main tree. The support of the Optimus technology is really important for users of all modern laptops. Please, commit it, at least for tested versions of Nvidia drivers.
With actual driver, I manage to use optimus with VirtualGL following https://wiki.freebsd.org/OptimusVideoSupport (for the xorg.conf.nv ) It is working with a little workaround: I have to rename libglx.so from /usr/local/lib/xorg/modules/extensions to something else : for example : notlibglxso , before starting a X session. In my kld_list I have i915kms, so that when X start it runs with intel driver. (but in that case it fails to get glx) . After I start a X session, I start by loading nvidia-modeset, I rename notlibglxso to its original name. And then I start another server using the nvidia card: X -conf xorg.conf.nv -sharevts -noreset :8 I can launch a program with the nvidia card with virtualGL /usr/local/VirtualGL/bin/vglrun -d ":8" program With glxinfo I get information about my NVIDIA card so I guess it is working
If the other can confirm, we could close this. Please, on all feedback, thank you!
(In reply to Thibault Payet from comment #19) I use a similar approach like yours and I can run applications on the nvidia if I start a second X server. However, with this approach you can't run applications on the first X server anymore which depend on libGL.so.1. This is because the Nvidia driver installation overrides this lib with the Nvidia version of it. If I understand the idea of the proposed patch correctly, this would solve the problem. I haven't had a chance to test this though due to the patch would need to receive some updates first to work with the current Nvidia driver installation.
Created attachment 196829 [details] WIP: Not working for Linux option, just an example This is my local version of nvidia-driver, it is the combinaison of this bug report patch, the bug #226403, and also some modification that I made to allow running OpenGL applications with the integrated graphics cards while we don't need the nvidia card, to accomplish that I haved to move etc/libmap.d/nvidia.conf to a hidden directory etc/libmap.d/.nvidia/nvidia.conf so that the linker will not try to link with the nvidia library Since the nvidia glx extension has moved to .nvidia_optimus/libglx.so, by doing kldload nvidia-modeset, the glx library of nvidia will not be moved to extensions/libglx.so . I use for that another module directory where I symlink the one on /usr/local/lib/xorg/modules (all except drivers and extensions). in extensions I have libglx.so a symlink to the nvidia one and in drivers I have a symlink of modesetting_drv.so nvidia_drv.so vesa_drv.so when I want to use the nvidia driver, I have to first cp etc/libmap.d/.nvidia/nvidia.conf to etc/libmap.d/nvidia.conf kldload the nvidia-modeset launching Xorg with nvidia config from the wiki https://wiki.freebsd.org/Graphics/OptimusVideoSupport X -config /etc/X11/xorg.conf.nvidia -configdir /etc/X11/xorg.conf.d -modulepath $NVIDIA_MODULE -sharevts -noreset :8 where NVIDIA_MODULE is the directory previously created that does the symlink with libglx.so from nvidia then I can launch an application on the nvidia card via VirtualGL When I have no more need to the card, I just have to kill the server, unload the nvidia-modeset, and then remove etc/libmap.d/nvidia.conf and then I can launch OpenGL application using the integrated graphics cards
It would be nice to see the port committed even if there are some problems with Linux compatibility level. Actually new users have impression that Optimus in not supported at all, so anyone who wants to install FreeBSD on a laptop would not do it, passing to Linux. It degrades seriously the image of FreeBSD and reduces the users base.
(In reply to peter from comment #23) Totally agree
Does any of the suggested methods work with a laptop that doesn't have an option to switch GPUs on BIOS?
(In reply to Luís Carneiro from comment #25) I basically use the approach described in comment #19 on my laptop. I use a Huawei Matebook X Pro which does not allow to turn on/off the graphic cards in the BIOS. Just to add. I turn off the Nvidia card via acpi_call (from ports) if I don't need it. That helps a lot to save power.
Is there still a plan to commit the changes to the port ? Or wll https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=232645 solve all this magically ?
This was opened in 2014 and it's still a work in progress, but oh well, here it goes... I'm working on an optimus port, it can be found here: https://github.com/pouya-eghbali/freebsd-nvidia-optimus Section A) How it works A little explanation about how it works, before we go into details about making a port or a PR: 1. We have some conflicting files and modules, the GL/EGL libraries and the Xorg extensions which Nvidia driver overwrites and makes the Intel GPU unusable. 2. We install these libraries and conflicting files in a different location. 3. We start an extra Xorg server for the Nvidia asking it to load extensions from the locations specified in step 2. 4. We run our programs on the step 3 Xorg server, and ask the program to load libraries from the location in step 2. All this can be done using VirtualGL. My port does the above, and installs services and utilities to work with Nvidia drivers, it also auto generates a config file for the step 2 Xorg server (extracting the bus id using pciconf). It provides an `optimus` service to control the X server (generate config, start, stop...) and load/unload Nvidia modules. It also provides a `optirun` utility, which uses VirtualGL to run the programs on the Nvidia GPU. Section B) how to port Now, about porting this or making a PR. There can be different approaches, let's go step by step. 1. For step two in section A, there can be different approaches. One way is to move the conflicting files to a different location, and name them correctly! (for example libGL instead of libGL-NVIDIA). We don't need to use libmap for optimus. Another approach is to move ALL nvidia files into a different location (not install in /usr/local/lib, but in /usr/local/lib/optimus for example), and also name them correctly. The latter approach makes it easier to port newer versions of Nvidia driver, because instead of moving files one by one we just move all of them. 2. For step three and four in section A, there can be different approaches. `primus` is old and dead (last updated 7 years ago), VirtualGL is old but not dead (last updated 6 days ago), and also VirtualGL is available in FreeBSD ports (So I decided to go that way with my port). Another possible solution is to do PRIME with randr as explained in Nvidia documentations (Not sure if that'll work with FreeBSD, need to test it). Anyways, seems like VirtualGL is the best we have at least for now. 3. For porting, either we clone the nvidia-driver package, modify for optimus and put everything in one port. This is what I did because this can be done quickly and gives me much more freedom, however it makes maintaining it more difficult. The other way is to add an optimus option to the nvidia-driver, then make an slave port with that option on, or just make an slave port and move the files after the build is done (not sure if that's possible, just an idea), this port will install the required Nvidia libraries/extensions/modules, and another port named optimus-utilities-${backend} (where backend can be primus, virtualgl or prime) will provide the necessary tools and utilities to work with the optimus Nvidia GPU. I would like to hear other people's opinions. nvidia-driver port is at v390, 430 is out and the PRs are pending. I would prefer to have 430 merged before we make a port for this, until then the Makefile and builds (the quick approach) is available on my github repository.
Further discussion: https://forums.freebsd.org/threads/nvidia-optimus-driver-for-freebsd.71504/ In particular, we are considering a multiple-port solution so that the generic parts aren't tied down to "optimus". Github containing this approach, working, but not updated to 430.x: https://github.com/therontarigo/freebsd-gpu-headless/ I would file a separate PR for this, but is reviews.freebsd.org better to use in this case?
Now that nvidia-driver ports have been refactored, can this be moved forward, danfe@ ?
(In reply to Gleb Popov from comment #30) Adding the hooks from https://github.com/therontarigo/freebsd-gpu-headless/blob/master/ports/x11_nvidia-driver.diff to the now-refactored nvidia-driver is the starting point. However, I can't promise any schedule by which I can complete this. danfe@, is this (just the nvidia-driver hooks) something you are comfortable taking over? In short, the list of requirements, to allow the optimus slave ports to get their work done: - Overridability of PORTNAME and COMMENT - Overridability of SUB_FILES since the slave won't want the pkg-*install scripts. - Move libglx.so into ${EXTENSIONSDIR} defaulting to ${MODULESDIR}/extensions/.nvidia (not moved) - Create ${LIBGLDIR} if defined, containing "libGL.so" linking to ${PREFIX}/lib/libGL-NVIDIA.so (likewise for each lib*GL*.so*) - Only install libmap.d/nvidia.conf if NO_LIBMAP is undefined Whether ${PREFIX}/lib/libGL-NVIDIA.so is libGLVND OR the actual Nvidia implementation *does not matter* to the operation of the remaining nvidia-headless ports.
Gleb, Theron, indeed, with Linuxish libs split in ports r515584 further changes to the driver itself are unblocked; I'm working on the update to 430.x as we speak. Once the dust settles (in a few days), we'd proceed with handling stuff like Optimus, etc.
Bump. Any progress on this? NVIDIA "headless" port is also required for CUDA, which I'm playing with ATM.
(In reply to Gleb Popov from comment #33) Hi, do you mind testing with 440 branch of https://github.com/therontarigo/freebsd-gpu-headless ? For my part in choosing to call this "headless" - that name has been bothering me. s/headless/secondary/ ? Once it is committed to ports tree the name won't be changed.
(In reply to Theron Tarigo from comment #34) It worked pretty much great! The only thing is I don't have nvrun-vgl, so I don't see any windows. glxgears, however, does run and prints incredible FPS.
(In reply to Gleb Popov from comment #35) Gleb, do you mean that nvrun-vgl was missing, or that you are on a headless server where it isn't wanted? For applications without graphical output, nvidia_xorg shouldn't technically be needed: Nvidia's EGL, Vk, and CUDA libs are intended as I understand to connect directly to the driver with Xorg only needed if a drawable framebuffer is created. Whether that actually works is another question. Maybe this is something you can look into?
(In reply to Theron Tarigo from comment #36) > Gleb, do you mean that nvrun-vgl was missing, or that you are on a headless server where it isn't wanted? The former, there is no nvrun-vgl executable.
(In reply to Gleb Popov from comment #37) What is output of ? $ pkg list nvidia-hybrid-graphics
(In reply to Theron Tarigo from comment #38) Ha, stupid me. I just forgot `make install` for nvidia-hybrid-graphics. Now nvrun-vgl glxgears show a window, however FPS number are very sad: 7 frames in 5.0 seconds = 1.397 FPS 5 frames in 5.0 seconds = 1.000 FPS
(In reply to Gleb Popov from comment #39) > FPS number are very sad: > 7 frames in 5.0 seconds = 1.397 FPS > 5 frames in 5.0 seconds = 1.000 FPS Sorry to hear you are affected by this as well, but helpful to know that I am not only one seeing this. Xorg appears to go into some sort of power-saving is my guess, since sending input to the server gets it to recover (for a while, at least): $ nvrun x11vnc -listen localhost & sleep 3 ; vncviewer localhost Then anything running under nvrun-vgl goes to full performance as soon as I click in the blank desktop. This didn't happen with 390.x drivers, I think I will need to dig through Nvidia release notes to see if there is a mention of anything relevant.
Neverthless, it is a huge step forward. danfe@ please find time to commit this.
(In reply to Gleb Popov from comment #41) > danfe@ please find time to commit this. I'm going to create a Phabricator review for this. The new ports, even if somewhat buggy, are a step forward, but there are also changes to Makefile of nvidia-driver itself, which makes this more sensitive since any mistake there could affect all existing Nvidia users.
Please see https://reviews.freebsd.org/D22521
Another 2 months passed. ping @danfe
(In reply to Gleb Popov from comment #44) Not danfe, but thanks for the bump. Of course I was hoping @danfe would review D22521 but it seems that may not happen. At least the attention has prompted me to see a path toward resolving one of the two outstanding issues I know of (legacy vs. current nvidia and FLAVOR for this). The other outstanding issue is the performance drop of the Nvidia server to 1FPS. It's a power-save "feature" that must be turned off (Option "HardDPMS" "false") for our use-case (at no significant impact to real power savings that I can tell). I wanted to include the line in the port's default xorg.conf but frustratingly nvidia-xconfig strips it out and I haven't thought of any workaround that isn't ugly.
(In reply to Theron Tarigo from comment #45) > nvidia-xconfig strips it As far as I can tell, nvidia-xconfig likes to move options from Device to Screen section(s).
danfe, now what about this? If you give some input I can do the rebasing/fixing/committing work.
Cough. Well, sorry to be bold, but I'm setting a maintainer-feedback flag and will commit https://reviews.freebsd.org/D22521 after timeout (2 weeks). I hope danfe won't get upset, but this is taking too long.
(In reply to Gleb Popov from comment #48) Thanks for taking action on this. Update to related port VirtualGL (bug #241660) needs attention as well, the maintainer indicated approval but that wasn't committed and now there are some additional patches.
Is there any further info regarding this port? Thanks, Ka Ho
(In reply to Ka Ho Ng from comment #50) I have been dog-fooding the patches for this for a couple weeks now and making heavy use of the nvidia GPU on my system (for both firefox and chrome) and it has been stable. I think it would be great if we could get this pushed across the finish line, but I am not aware of what other outstanding issues need to be addressed.
ping @danfe, now that https://reviews.freebsd.org/D22521 is updated?
^Triage: - This is a new port request (as reported), assignee timeout (i'll take this to coordinate). @Gleb Are you still able to take a new port or or nvidia port change set to resolution? Additional Details: *Technically*, this issue (as reported), for a new port sub-ported off another port: a) Does not require maintainer approval (though its great to get feedback/advice) b) was not updated to reflect the integration into the main nvidia ports later in this issues history, which does require maintainer approval. The review within which this took place was added only as a comment (not in URL field or as an attachment), so technically could not be approved (maintainer-approval flag on the attachment) Technically, maintainer timeout's don't apply to or on reviews alone, and coupled with (b) has made this issue appear more difficult and complicated to resolve/land than it needed to be. At this stage, there is an opportunity to resolve this issue *quickly* in two ways: * Anyone to immediately land a *new* port *sub-ported* off nvidia driver ports, not requiring maintainer approval, with an updated and QA'd patch. * Attach review D22521 patch here as an attachment, and it times out in 14 days and gets implicit maintainer approval. The above two options will ensure the least likelihood of anyone getting upset or leveraging technicalities that aren't particularly useful to getting what is now an almost a 7 year old issue resolved. Let's get this done.
Created attachment 224760 [details] D22521: new port x11/nvidia-hybrid-graphics, non-functional changes to x11/nvidia-driver and x11/nvidia-settings > Anyone to immediately land a *new* port *sub-ported* off nvidia driver ports Not going to happen without the nonfunctional changes to x11/nvidia-driver itself; as it currently exists it is not friendly to the relevant slave port (duplicating much of nvidia-driver just to avoid cooperation seems undesirable). > Attach review D22521 patch here Done. I'm not sure it's my role here to mark obsolete the older efforts.
Maintainer timeout?
^Triage: Update summary to reflect D22521
Comment on attachment 224760 [details] D22521: new port x11/nvidia-hybrid-graphics, non-functional changes to x11/nvidia-driver and x11/nvidia-settings Pending QA: Approved by: portmgr (maintainer timeout: 15 days)
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/ports/commit/?id=2ee6acf85a98de36269d3a727b4c45675b0eb9c3 commit 2ee6acf85a98de36269d3a727b4c45675b0eb9c3 Author: Theron Tarigo <theron.tarigo@gmail.com> AuthorDate: 2021-06-15 19:27:11 +0000 Commit: Kevin Bowling <kbowling@FreeBSD.org> CommitDate: 2021-06-15 19:30:22 +0000 x11/nvidia-hybrid-graphics: Optimus Technology PR: 192617 Reported by: David Mackay <davidjx8p@gmail.com> Reviewed by: many Tested by: many Approved by: portmgr (maintainer timeout: 15 days) Differential Revision: https://reviews.freebsd.org/D22521 x11/nvidia-driver/Makefile | 50 ++++++++++++-- x11/nvidia-driver/pkg-plist | 9 ++- x11/nvidia-hybrid-graphics/Makefile (new) | 78 ++++++++++++++++++++++ .../files/nvidia_xorg.in (new) | 48 +++++++++++++ .../files/pkg-message.in (new) | 24 +++++++ .../files/src/bin/Xorg-nvidia-headless.in (new) | 17 +++++ .../files/src/bin/nvidia-headless-xconfig.in (new) | 9 +++ .../files/src/bin/nvrun-vgl.in (new) | 8 +++ .../files/src/bin/nvrun.in (new) | 7 ++ .../xorg-nvidia-headless-template.conf.in (new) | 39 +++++++++++ .../files/src/etc/nvidia-headless.conf.in (new) | 1 + .../files/src/etc/nvidia-hybrid.conf.in (new) | 1 + .../nvidia-headless-utils/readconf.in (new) | 5 ++ .../src/libexec/nvidia-settings-hybrid.in (new) | 4 ++ x11/nvidia-hybrid-graphics/pkg-descr (new) | 8 +++ x11/nvidia-hybrid-graphics/pkg-plist (new) | 11 +++ x11/nvidia-secondary-driver-390/Makefile (new) | 6 ++ x11/nvidia-secondary-driver/Makefile (new) | 20 ++++++ x11/nvidia-secondary-driver/pkg-message (new) | 4 ++ x11/nvidia-settings/Makefile | 7 ++ x11/nvidia-settings/files/nvidia-settings.in (new) | 3 + 21 files changed, 352 insertions(+), 7 deletions(-)
Thanks to everyone for all the work that went into this!
I don't see why portmgr needed to approve this?