When installing x11/nvidia-driver and selecting the LINUX configuration option only 32-bit libraries are installed in /compat/linux/lib/. linux-c7 and FreeBSD 11 support 64-bit Linux and thus 64-bit libraries should ideally be installed, where supported, in /compat/linux/lib64/ for compatibility with 64-bit applications. I would propose checking for the existence of /compat/linux/lib64 as the trigger for installing the lib64 libraries. Thanks.
*** Bug 219698 has been marked as a duplicate of this bug. ***
Yes, this is really important! However, I think it is not only an issue of missing libraries, but an actual driver issue. I have tried putting the right libraries in place, but it doesn't work. However, more and more linux binaries are now 64bit (only) so this is a real problem, rendering the linux compatibility useless for very many apps, including, but not limited to all modern games.
Created attachment 199099 [details] patch This patch changes the x11/nvidia-driver ports to fetch and install the Linux driver package under /compat/linux. This includes both 32 bit and 64 bit libraries, unlike the Linux libraries included in the FreeBSD driver package which are only 32 bit. The 64 bit libraries require at least r340259 of FreeBSD stable/11, stable/12 or head. I've been able to test x11/nvidia-driver-304 successfully, but I cannot test the other versions. The Linux libraries in x11/nvidia-driver are libglvnd based and I'm not sure that will work, so it would be nice if somebody could test that before this patch is committed.
(In reply to Tijl Coosemans from comment #3) > The Linux libraries in x11/nvidia-driver are libglvnd based The Linux driver provides both libglvnd compatible and legacy GL libraries. See https://devtalk.nvidia.com/default/topic/915640/multiple-glx-client-libraries-in-the-nvidia-linux-driver-installer-package/ and http://us.download.nvidia.com/XFree86/Linux-x86_64/390.87/README/installedcomponents.html.
Thank you for working on this, it is really appreciated! Do these changes make the driver compatible with linux_base-c7, too? I think the main issue are changed paths (/usr is a symlink now) which broke stuff for me on install/uninstall some times...
(In reply to Hannes Hauswedell from comment #5) I haven't tried linux_base-c7, but /usr isn't a symlink there. /(s)bin and /lib(64) are, but that's also the case with linux_base-c6, so that shouldn't be a problem.
I have updated to 13-BETA4, removed all -c7 ports, installed linux_base-c6, applied the patch here (which applied cleanly), installed x11/nvidia-driver (nvidia-driver-390.87). But running /compat/linux/usr/bin/glxinfo leads to immediate dump of the kernel :'( I have made the dump available here: https://hauswedell.net/crash/core.tar.zst I have a GTX 750 Ti, I think I can't try any of the older drivers... Thanks for your help!
Created attachment 199205 [details] patch3 The linux ioctl handler in the nvidia driver didn't wrap the FreeBSD ioctl handler properly. This patch should fix that.
(In reply to Tijl Coosemans from comment #8) Thanks, this fixes the coredump! But now I run into #228287 , i.e. I get % /compat/linux/usr/bin/glxinfo name of display: unix:0.0 X Error of failed request: BadValue (integer parameter out of range for operation) Major opcode of failed request: 154 (GLX) Minor opcode of failed request: 3 (X_GLXCreateContext) Value in failed request: 0x0 Serial number of failed request: 25 Current serial number in output stream: 26 I tried setting IndirectRendering as described in #228287, but then OpenGL-Apps won't start either because they can't allocate resources. I assume IndirectRendering deactivates proper 3D anyway?
(In reply to Hannes Hauswedell from comment #9) What is the output of "/compat/linux/bin/sh ldd /bin/glxinfo"?
(In reply to Tijl Coosemans from comment #10) % /compat/linux/bin/sh ldd /bin/glxinfo linux_vdso.so.1 => (0x00007fffffffe000) libGLU.so.1 => /lib64/libGLU.so.1 (0x0000000801400000) libGL.so.1 => /lib64/libGL.so.1 (0x0000000801800000) libm.so.6 => /lib64/libm.so.6 (0x0000000801c00000) libX11.so.6 => /lib64/libX11.so.6 (0x0000000802000000) libXext.so.6 => /lib64/libXext.so.6 (0x0000000802400000) libc.so.6 => /lib64/libc.so.6 (0x0000000802800000) libstdc++.so.6 => /lib64/libstdc++.so.6 (0x0000000802c00000) libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x0000000803000000) libdl.so.2 => /lib64/libdl.so.2 (0x0000000803400000) libGLX.so.0 => /lib64/libGLX.so.0 (0x0000000803800000) libGLdispatch.so.0 => /lib64/libGLdispatch.so.0 (0x0000000803c00000) libxcb.so.1 => /lib64/libxcb.so.1 (0x0000000804000000) /lib64/ld-linux-x86-64.so.2 (0x0000000001021000) libXau.so.6 => /lib64/libXau.so.6 (0x0000000804400000)
(In reply to Hannes Hauswedell from comment #11) Please run the following commands and attach ktrace.txt to this bug. ktrace -i -f /where/you/want/ktrace.out /compat/linux/bin/glxinfo kdump -H -f /where/you/want/ktrace.out > /where/you/want/ktrace.txt
Created attachment 199213 [details] ktrace The file is attached, thanks for debugging this!
(In reply to Hannes Hauswedell from comment #13) Looks like a problem with the nvidia-kms module. Can you install graphics/mesa-demos and create a ktrace of the FreeBSD glxinfo? Can you also give the non-kms module a try?
Created attachment 199218 [details] native ktrace This is the ktrace from the native glxinfo. I tried booting with the nvidia-module instead of nvidia-modeset, but then I never make it into X. There are no warnings or errors in Xorg.0.log or dmesg, it just hangs when switching resolution. (It's possible to stop/restart X via ssh, but also with startx I get no suspicious output, just a black screen in console-resolution with white cursor). I remember having had this a long time ago already which is why I switched to the kms-module in the first place, I think.
Created attachment 199220 [details] redev_test This test triggers a slightly different codepath in libGL where it doesn't try to recreate /nvidia* device files. This seems to be enough to get glxgears working for me with 415.13 driver. Try it with: % /compat/linux/bin/cc -m64 -std=c99 -shared -fPIC -lm -pthread -I/compat/linux/usr/include -L/compat/linux/lib64 -o test.so stat_rdev_override.c % env LD_PRELOAD=$PWD/test.so /compat/linux/bin/glxgears Does that help?
(In reply to Alex S from comment #16) What's the recommended way to get a compiler into the linux base? There seem to be no linux-gcc packages on FreeBSD and inside the compat there isn't even rpm to install original CentOS packages from a mirror... Also you mentionened an nvidia driver of version 415.13. Where can I get that and does it work with the patches from this PR? Thanks for your help with this!
(In reply to Hannes Hauswedell from comment #17) There's a compiler in devel/linux-c6-devtools.
Ah, ok. Why don't they just call it GCC like every one else :| Anyway, preloading the above object while using nvidia-driver-390.87 leads to another coredump: https://hauswedell.net/crash/core4.tar.zst
> Also you mentioned an nvidia driver of version 415.13. Well, a knee-jerk patchjob on top of existing port takes about 10 min or so. However, 410.57+ actually requires libglvnd to work (a compiled copy is provided with the driver) and it really should be packaged separately. Somebody should look into making libglvnd work with Mesa as well. > Where can I get that and does it work with the patches from this PR? I'm actually using 11.2-RELEASE and a slightly modified patch from PR 206711 (too lazy to bother with compiling whole kernel), but generally the latest driver should work with these patches as well, there is no reason not to. > Anyway, preloading the above object while using nvidia-driver-390.87 leads to another coredump linux_sendmsg_common? Looks like a separate bug to me.
Hm, what are the next steps? I had the feeling we were making progress! Anything I can do to help move this forward? Should I open a different PR with the coredump? Will the patches here be merged?
(In reply to Hannes Hauswedell from comment #21) The patch for the second panic is currently under review.
Created attachment 199335 [details] linux.sendmsg.patch This patch should fix the second panic. You can test it as follows: cd /usr/src svn patch /path/to/linux.sendmsg.patch cd sys/modules/linux make make install cd ../linux64 make make install mv /boot/modules/linux.ko /boot/kernel/ mv /boot/modules/linux64.ko /boot/kernel/ kldxref /boot/modules kldxref /boot/kernel shutdown -r now
Created attachment 199336 [details] patch4 This version of the patch for x11/nvidia-driver adds an extra ktrace point to the nvidia-modeset driver. Can you create another ktrace of both the Linux and FreeBSD glxinfo?
A commit references this bug: Author: tijl Date: Mon Nov 19 15:31:55 UTC 2018 New revision: 340631 URL: https://svnweb.freebsd.org/changeset/base/340631 Log: Do proper copyin of control message data in the Linux sendmsg syscall. Instead of calling m_append with a user address, allocate an mbuf cluster and copy data into it using copyin. For the SCM_CREDS case, instead of zeroing a stack variable and appending that to the mbuf, zero part of the mbuf cluster directly. One mbuf cluster is also the size limit used by the FreeBSD sendmsg syscall (uipc_syscalls.c:sockargs()). PR: 217901 Reviewed by: kib MFC after: 3 days Changes: head/sys/compat/linux/linux_socket.c
I have applied the kernel patch and the new nvidia patch. Linux glxinfo still produces: name of display: unix:0.0 X Error of failed request: BadValue (integer parameter out of range for operation) Major opcode of failed request: 154 (GLX) Minor opcode of failed request: 3 (X_GLXCreateContext) Value in failed request: 0x0 Serial number of failed request: 25 Current serial number in output stream: 26 I will add updated ktraces.
Created attachment 199359 [details] Linux compat ktrace of glxinfo
Created attachment 199360 [details] native ktrace of glxinfo
(In reply to Hannes Hauswedell from comment #27) > 1550 100663 glxinfo NAMI "/compat/linux/dev/nvidiactl" > 1550 100663 glxinfo NAMI "/dev/nvidiactl" > 1550 100663 glxinfo RET linux_unlink -1 errno -13 Permission denied You forgot to test this with LD_PRELOAD trick.
(In reply to Alex S from comment #29) Yes, but doing that still results in a coredump for me: https://hauswedell.net/crash/core6.tar.zst BTW: I had problems with updating the module as described above, after just copying the module the nvidia driver claimed that some things were of different versions. So I updated to 12-stable, applied the patch, rebuilt and installled entire world and kernel. I assume that's supposed to work right, too?
A commit references this bug: Author: tijl Date: Tue Nov 20 14:18:57 UTC 2018 New revision: 340674 URL: https://svnweb.freebsd.org/changeset/base/340674 Log: Fix another user address dereference in linux_sendmsg syscall. This was hidden behind the LINUX_CMSG_NXTHDR macro which dereferences its second argument. Stop using the macro as well as LINUX_CMSG_FIRSTHDR. Use the size field of the kernel copy of the control message header to obtain the next control message. PR: 217901 MFC after: 2 days X-MFC-With: r340631 Changes: head/sys/compat/linux/linux_socket.c
Created attachment 199382 [details] linux.sendmsg.2.patch Patch for the second linux_sendmsg panic. To be applied on top of the first patch.
Does the LD_PRELOAD method still panic?
I wasn't at the affected computer the last days, but I have now verified that everything works with the second kernel patch applied! After building another test.so with -m32 I can successfully start linux OpenGL in either 32bit or 64bit by prefixing the respective shared object file. Even "modern" programs like linux-unigine-valley (whose port I maintain) work. Very nice, thanks for debugging this :) Now we only need to make linux-base-c7 default and I can remove the dual-boot. Should I mark this resolved now, or do we wait till everything has landed in ports and Kernel?
(In reply to Hannes Hauswedell from comment #34) Thanks for testing. I'll update the patch so you don't need to preload anything. We'll close the bug when everything is committed.
Thanks! I am busy testing applications right now, is there a place I should report failing applications that I know work on Linux? It's currently hit and miss, roughly half work flawlessly, some crash on start, some hang...
A commit references this bug: Author: tijl Date: Thu Nov 22 09:41:46 UTC 2018 New revision: 340755 URL: https://svnweb.freebsd.org/changeset/base/340755 Log: MFC r340631: Do proper copyin of control message data in the Linux sendmsg syscall. Instead of calling m_append with a user address, allocate an mbuf cluster and copy data into it using copyin. For the SCM_CREDS case, instead of zeroing a stack variable and appending that to the mbuf, zero part of the mbuf cluster directly. One mbuf cluster is also the size limit used by the FreeBSD sendmsg syscall (uipc_syscalls.c:sockargs()). PR: 217901 Reviewed by: kib Changes: _U stable/12/ stable/12/sys/compat/linux/linux_socket.c
A commit references this bug: Author: tijl Date: Thu Nov 22 09:41:54 UTC 2018 New revision: 340756 URL: https://svnweb.freebsd.org/changeset/base/340756 Log: MFC r340631: Do proper copyin of control message data in the Linux sendmsg syscall. Instead of calling m_append with a user address, allocate an mbuf cluster and copy data into it using copyin. For the SCM_CREDS case, instead of zeroing a stack variable and appending that to the mbuf, zero part of the mbuf cluster directly. One mbuf cluster is also the size limit used by the FreeBSD sendmsg syscall (uipc_syscalls.c:sockargs()). PR: 217901 Reviewed by: kib Changes: _U stable/11/ stable/11/sys/compat/linux/linux_socket.c
A commit references this bug: Author: tijl Date: Thu Nov 22 09:47:43 UTC 2018 New revision: 340757 URL: https://svnweb.freebsd.org/changeset/base/340757 Log: MFC r340674: Fix another user address dereference in linux_sendmsg syscall. This was hidden behind the LINUX_CMSG_NXTHDR macro which dereferences its second argument. Stop using the macro as well as LINUX_CMSG_FIRSTHDR. Use the size field of the kernel copy of the control message header to obtain the next control message. PR: 217901 Changes: _U stable/12/ stable/12/sys/compat/linux/linux_socket.c
A commit references this bug: Author: tijl Date: Thu Nov 22 09:47:52 UTC 2018 New revision: 340758 URL: https://svnweb.freebsd.org/changeset/base/340758 Log: MFC r340674: Fix another user address dereference in linux_sendmsg syscall. This was hidden behind the LINUX_CMSG_NXTHDR macro which dereferences its second argument. Stop using the macro as well as LINUX_CMSG_FIRSTHDR. Use the size field of the kernel copy of the control message header to obtain the next control message. PR: 217901 Changes: _U stable/11/ stable/11/sys/compat/linux/linux_socket.c
A commit references this bug: Author: tijl Date: Thu Nov 22 13:09:43 UTC 2018 New revision: 340761 URL: https://svnweb.freebsd.org/changeset/base/340761 Log: MFS r340755: Do proper copyin of control message data in the Linux sendmsg syscall. Instead of calling m_append with a user address, allocate an mbuf cluster and copy data into it using copyin. For the SCM_CREDS case, instead of zeroing a stack variable and appending that to the mbuf, zero part of the mbuf cluster directly. One mbuf cluster is also the size limit used by the FreeBSD sendmsg syscall (uipc_syscalls.c:sockargs()). PR: 217901 Reviewed by: kib Approved by: re (marius) Changes: _U releng/12.0/ releng/12.0/sys/compat/linux/linux_socket.c
A commit references this bug: Author: tijl Date: Thu Nov 22 13:12:17 UTC 2018 New revision: 340762 URL: https://svnweb.freebsd.org/changeset/base/340762 Log: MFS r340757: Fix another user address dereference in linux_sendmsg syscall. This was hidden behind the LINUX_CMSG_NXTHDR macro which dereferences its second argument. Stop using the macro as well as LINUX_CMSG_FIRSTHDR. Use the size field of the kernel copy of the control message header to obtain the next control message. PR: 217901 Approved by: re (marius) Changes: _U releng/12.0/ releng/12.0/sys/compat/linux/linux_socket.c
Created attachment 199451 [details] patch5 This version of the patch for x11/nvidia-driver adds calls to linux_device_register_handler so the kernel reports the right device major and minor numbers to linux programs. You should no longer need to preload anything.
(In reply to Hannes Hauswedell from comment #36) > I am busy testing applications right now, is there a place I should report > failing applications that I know work on Linux? It's currently hit and miss, > roughly half work flawlessly, some crash on start, some hang... Well, let's get the nvidia-driver working first. Then you can file bugs for applications that fail. Some general tips: Make sure you have tmpfs mounted on /compat/linux/dev/shm, linprocfs on /compat/linux/proc, and linsysfs on /compat/linux/sys. Run Linux shell scripts using /compat/linux/bin/sh, not /bin/sh. You can check for missing libraries with "/compat/linux/bin/sh ldd /path/to/program_or_library". Applications that need to be installed are best installed under /compat/linux (e.g. /compat/linux/opt). You can then put a little startup script in /usr/local/bin that looks like this: #! /bin/sh exec /compat/linux/path/to/program "$@" Or this if program is a shell script: #! /bin/sh exec /compat/linux/bin/sh /path/to/program "$@"
Have you been able to test patch5?
(In reply to Tijl Coosemans from comment #45) Yes, the patch makes glxinfo, glxgears and at least Pillars of Eternity work on my system without preloading anything. I had recently also updated to linux_base-c7 which broke all unigine-engine based games/benchmarks, but I suspect that's unrelated. Here's my current status: Works with patch, both -c6 and -c7: Pillars Of Eternity (64Bit) [no sound, though] games/linux-doom3-demo (32Bit) Works with patch, but only linux_base-c6, not linux_base-c7: games/linux-unigine-valley (64Bit) OilRush (64Bit) [both segfault on -c7] Works with neither, application crash (segfault or sigabrt): Dungeons 2 (64Bit GOG) Torment: Tides of Numenera (64Bit GOG) Works with neither, application hang: Dungeons 3 (64Bit GOG) games/linux-quake4-demo (32Bit) [not tested on -c6] So all-in-all, I strongly support merging this patch, but I would be really grateful if we could work on debugging some things further as currently most linux-apps don't work, yet. Thanks for all the time invested into this!
Alexey, I think patch5 (generated using --show-copies-as-adds) is ready to be committed (minus PORTREVISION bumps). It does not include nvidia-uvm like earlier patches because that's out of scope for now. Below is a detailed description. Can you look into it, suggest changes, approve it? M Makefile M distinfo D files/extra-patch-src-Makefile > moved to files/extra-patch-src-Makefile.in A + files/extra-patch-src-Makefile.in > moved from files/extra-patch-src-Makefile D files/extra-patch-src-nv-freebsd.h > moved to files/extra-patch-src-nv-freebsd.h.in A + files/extra-patch-src-nv-freebsd.h.in > moved from files/extra-patch-src-nv-freebsd.h D files/extra-patch-src-nv-misc.h > moved to files/extra-patch-src-nv-misc.h.in A + files/extra-patch-src-nv-misc.h.in > moved from files/extra-patch-src-nv-misc.h A files/extra-patch-src-nvidia_ctl.c.in A files/extra-patch-src-nvidia_dev.c.in A files/extra-patch-src-nvidia_linux.c.in A files/extra-patch-src-nvidia_pci.c.in M files/extra-patch-src_nvidia-modeset_nvidia-modeset-freebsd.c D files/extra-patch-src_nvidia_Makefile D files/extra-patch-src_nvidia_nv-freebsd.h D files/extra-patch-src_nvidia_nv-misc.h M pkg-plist Log: Install both 32 bit and 64 bit Linux libraries from the official Linux driver distribution instead of the ones bundled with the FreeBSD driver which are incomplete and 32 bit only. This makes amd64 Linux OpenGL programs work on FreeBSD 12.0 and recent 11-STABLE. This does not enable CUDA. - Initialise DISTFILES so a file can be added by the LINUX option. - Set EXTRACT_ONLY because the file added by the LINUX option is a self-extracting archive handled in post-extract. - The patches for <355.06 and >=355.06 are identical except for the path. Keep only one set (listed in SUB_PATCHES) because more patches have been added. Modify their path in pre-patch. The patches are renamed from extra-patch-* to extra-patch-*.in. - Set OPTIONS_SUB and use option helpers for the LINUX option. - Define LINUX_ARCH_SUFX like ARCH_SUFX because LINUX_ARCH can be i386 on amd64. - Set LINUX_LIBDIR to the main lib directory: (/compat/linux)/usr/lib on i386 and (/compat/linux)/usr/lib64 on amd64. (The compat32 directory is (/compat/linux)/usr/lib.) - Define LINUX_LIBS, LINUX_LINKS, LINUX32_LIBS, LINUX32_LINKS, LINUX_PROGS, LINUX_MANS, and LINUX_DATA for all driver versions from 304.137 to 390.87. LINUX_LIBS and LINUX32_LIBS contain main libraries (i386/amd64) and compat32 libraries respectively. These are installed with INSTALL_LIB in post-install. LINUX_LINKS and LINUX32_LINKS contain symbolic link sources and their target. LINUX_PROGS are binaries installed with INSTALL_PROGRAM. LINUX_MANS are manuals installed with INSTALL_MAN. LINUX_DATA are data files and their destination, installed with INSTALL_DATA. The .if blocks for the different versions are sorted alphabetically on the first file. - Define PLIST_RE for all versions. These are sed commands applied to TMPPLIST in post-install. - Include bsd.port.pre.mk instead of bsd.port.options.mk to obtain LINUX_ARCH. - Adjust post-patch to disable installation of Linux libraries bundled with the FreeBSD driver. - Add post-patch-LINUX-on to setup nvidia_icd.json like nvidia-installer does on Linux. - Adjust existing post-install TMPPLIST edits to apply only to non-Linux files. - Add post-install-LINUX-on to install the Linux files. - Add files/extra-patch-src-nvidia_ctl.c.in and extra-patch-src-nvidia_dev.c.in. These patches make Linux stat(2) return the right major and minor number for /dev/nvidiactl and /dev/nvidia[0-9]+. Recent versions of the Linux libraries verify these. [1] Define the struct passed to linux_device_register_handler on the stack in nvidia_*_attach instead of as a global variable in case multiple graphics cards are attached in parallel. - Add files/extra-patch-src-nvidia_linux.c.in. It rewrites the Linux ioctl handler. The current implementation does not copyin arguments leading to kernel panics on FreeBSD 12 on machines that support Supervisor Mode Access Prevention (SMAP). - Add files/extra-patch-src-nvidia_pci.c.in. It adds a dependency on the linux_common kernel module for linux_device_register_handler. - Modify files/extra-patch-src_nvidia-modeset_nvidia-modeset-freebsd.c to add major/minor numbers (global struct here, because it is constant), rewrite ioctl, and add a dependency on linux_common for /dev/nvidia-modeset. PR: 217901 Analysed by: Alex S <iwtcex@gmail.com> [1] Tested by: Hannes Hauswedell <h2+fbsdports@fsfe.org> (390.87) Tested by: tijl (304.137)
> These patches make Linux stat(2) return the right major and minor number for /dev/nvidiactl and /dev/nvidia[0-9]+. Recent versions of the Linux libraries verify these. A minor nitpick: all versions of the driver, including legacy, have this check. Disabling the check through "ModifyDeviceFiles: 0" setting (should be placed in /proc/driver/nvidia/params) or faking successful remove/chmod/chown/xmknod sequence seems to work as well, so the new thing there is some kind of regression on some fallback path. It's not a deliberate feature.
(In reply to Alex S from comment #48) Hmm, with version 304 I don't see calls to stat(2) or unlink(2).
(In reply to Tijl Coosemans from comment #49) Looks like this particular check was introduced in addition to nvidia-modprobe utility in 319.12. Compare https://download.nvidia.com/XFree86/Linux-x86_64/304.137/README/faq.html#devicenodes against https://download.nvidia.com/XFree86/Linux-x86_64/319.12/README/faq.html#devicenodes.
A commit references this bug: Author: tijl Date: Fri Dec 14 16:24:05 UTC 2018 New revision: 487446 URL: https://svnweb.freebsd.org/changeset/ports/487446 Log: Install both 32 bit and 64 bit Linux libraries from the official Linux driver distribution instead of the ones bundled with the FreeBSD driver which are incomplete and 32 bit only. This makes amd64 Linux OpenGL programs work on FreeBSD 12.0 and recent 11-STABLE. This does not enable CUDA. - Initialise DISTFILES so a file can be added by the LINUX option. - Set EXTRACT_ONLY because the file added by the LINUX option is a self-extracting archive handled in post-extract. - The patches for <355.06 and >=355.06 are identical except for the path. Keep only one set (listed in SUB_PATCHES) because more patches have been added. Modify their path in pre-patch. The patches have been renamed from extra-patch-* to extra-patch-*.in. - Set OPTIONS_SUB and use option helpers for the LINUX option. - Define LINUX_ARCH_SUFX like ARCH_SUFX because LINUX_ARCH can be i386 on amd64. - Set LINUX_LIBDIR to the main lib directory: (/compat/linux)/usr/lib on i386 and (/compat/linux)/usr/lib64 on amd64. (The compat32 directory is (/compat/linux)/usr/lib.) - Define LINUX_LIBS, LINUX_LINKS, LINUX32_LIBS, LINUX32_LINKS, LINUX_PROGS, LINUX_MANS, and LINUX_DATA for all driver versions from 304.137 to 390.87. LINUX_LIBS and LINUX32_LIBS contain main libraries (i386/amd64) and compat32 libraries respectively. These are installed with INSTALL_LIB in post-install. LINUX_LINKS and LINUX32_LINKS contain symbolic link sources and their target. LINUX_PROGS are binaries installed with INSTALL_PROGRAM. LINUX_MANS are manuals installed with INSTALL_MAN. LINUX_DATA are data files and their destination, installed with INSTALL_DATA. The .if blocks for the different versions are sorted alphabetically on the first file. - Define PLIST_RE for all versions. These are sed commands applied to TMPPLIST in post-install. - Include bsd.port.pre.mk instead of bsd.port.options.mk to obtain LINUX_ARCH. - Adjust post-patch to disable installation of Linux libraries bundled with the FreeBSD driver. - Add post-patch-LINUX-on to setup nvidia_icd.json like nvidia-installer does on Linux. - Adjust existing post-install TMPPLIST edits to apply only to non-Linux files. - Add post-install-LINUX-on to install the Linux files. - Add files/extra-patch-src-nvidia_ctl.c.in and extra-patch-src-nvidia_dev.c.in. These patches make Linux stat(2) return the right major and minor number for /dev/nvidiactl and /dev/nvidia[0-9]+. Recent versions of the Linux libraries verify these. [1] Define the struct passed to linux_device_register_handler on the stack in nvidia_*_attach instead of as a global variable in case multiple graphics cards are attached in parallel. - Add files/extra-patch-src-nvidia_linux.c.in. It rewrites the Linux ioctl handler. The current implementation does not copyin arguments leading to kernel panics on FreeBSD 12 on machines that support Supervisor Mode Access Prevention (SMAP). - Add files/extra-patch-src-nvidia_pci.c.in. It adds a dependency on the linux_common kernel module for linux_device_register_handler. - Modify files/extra-patch-src_nvidia-modeset_nvidia-modeset-freebsd.c to add major/minor numbers (global struct here, because it is constant), rewrite ioctl, and add a dependency on linux_common for /dev/nvidia-modeset. PR: 217901 Analysed by: Alex S <iwtcex@gmail.com> [1] Tested by: Hannes Hauswedell <h2+fbsdports@fsfe.org> (390.87) Tested by: tijl (304.137) Approved by: maintainer timeout (15 days) Changes: head/x11/nvidia-driver/Makefile head/x11/nvidia-driver/distinfo head/x11/nvidia-driver/files/extra-patch-src-Makefile head/x11/nvidia-driver/files/extra-patch-src-Makefile.in head/x11/nvidia-driver/files/extra-patch-src-nv-freebsd.h head/x11/nvidia-driver/files/extra-patch-src-nv-freebsd.h.in head/x11/nvidia-driver/files/extra-patch-src-nv-misc.h head/x11/nvidia-driver/files/extra-patch-src-nv-misc.h.in head/x11/nvidia-driver/files/extra-patch-src-nvidia_ctl.c.in head/x11/nvidia-driver/files/extra-patch-src-nvidia_dev.c.in head/x11/nvidia-driver/files/extra-patch-src-nvidia_linux.c.in head/x11/nvidia-driver/files/extra-patch-src-nvidia_pci.c.in head/x11/nvidia-driver/files/extra-patch-src_nvidia-modeset_nvidia-modeset-freebsd.c head/x11/nvidia-driver/files/extra-patch-src_nvidia_Makefile head/x11/nvidia-driver/files/extra-patch-src_nvidia_nv-freebsd.h head/x11/nvidia-driver/files/extra-patch-src_nvidia_nv-misc.h head/x11/nvidia-driver/pkg-plist head/x11/nvidia-driver-304/Makefile head/x11/nvidia-driver-340/Makefile