Bug 244877 - graphics/mesa-dri: Fix vulkan by switching to unsynchronized userptr ioctls
Summary: graphics/mesa-dri: Fix vulkan by switching to unsynchronized userptr ioctls
Status: Open
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Some People
Assignee: freebsd-x11 (Nobody)
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2020-03-18 00:57 UTC by Austin Shafer
Modified: 2023-11-21 15:38 UTC (History)
4 users (show)

See Also:
bugzilla: maintainer-feedback? (x11)


Attachments
mesa-dri: patch for anv_gem.c (894 bytes, patch)
2020-03-18 00:57 UTC, Austin Shafer
no flags Details | Diff
mesa-dri: patch for anv_gem.c (1.19 KB, patch)
2020-03-18 17:43 UTC, Austin Shafer
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Austin Shafer 2020-03-18 00:57:35 UTC
Created attachment 212484 [details]
mesa-dri: patch for anv_gem.c

This is the userspace portion of a change which fixes the Vulkan graphics API on older Intel machines:

https://github.com/FreeBSDDesktop/kms-drm/pull/205

The above PR fixes the unsychronized DRM_IOCTL_I915_GEM_USERPTR which is used by intel's vulkan code. For the fix to be used the ioctl needs to have the I915_USERPTR_UNSYNCHRONIZED flag enabled:

    struct drm_i915_gem_userptr userptr = {
       .user_ptr = (__u64)((unsigned long) mem),
       .user_size = size,
-      .flags = 0,
+      .flags = I915_USERPTR_UNSYNCHRONIZED,
    };

    int ret = anv_ioctl(device->fd, DRM_IOCTL_I915_GEM_USERPTR, &userptr);

I've needed both the PR and this change for some daily vulkan development to work.

It seems that maybe all the mesa stuff is going to be overhauled soon? So maybe this isn't the ideal time to merge this. Please let me know if there is anything I can help with!
Comment 1 Jan Beich freebsd_committer freebsd_triage 2020-03-18 04:28:33 UTC
Maybe keep trying to use userptr a la attachment 187401 [details] (ignore timeout part obsoleted by review D23846). Doing so would make it possible to submit upstream[1] in order to help other BSDs.

[1] Confirm https://github.com/myfreeweb/freebsd-ports-dank/tree/lite/graphics/mesa-dev is still affected then rebase. Maybe apply in iris driver as well for consistency.
Comment 2 Jan Beich freebsd_committer freebsd_triage 2020-03-18 04:31:47 UTC
Other ports also try regular userptr then fall back to I915_USERPTR_UNSYNCHRONIZED:

$ ls -1d */*/files/*userptr*
lang/intel-compute-runtime/files/patch-userptr
multimedia/libva-intel-media-driver/files/patch-userptr
Comment 3 Jan Beich freebsd_committer freebsd_triage 2020-03-18 04:40:53 UTC
When I915_USERPTR_UNSYNCHRONIZED fails it maybe nice to advise user to run as root e.g.,

  if (geteuid() != 0) {
    fprintf(stderr, "%s", "ioctl(I915_GEM_USERPTR) failed. Try running as root but expect poor stability.\n");
  }

where "poor stability" alludes to markj@ comment in https://github.com/FreeBSDDesktop/kms-drm/issues/197#issuecomment-289590556

   That definition only allows unsynchronized userptr objects, and
   such objects can probably be used to trigger nasty system behaviour
   since there's nothing preventing the process from unmapping pages
   bound to a GEM userptr object.

However, I915_USERPTR_UNSYNCHRONIZED seems to work fine in compute-runtime, media-driver and xf86-video-intel.
Comment 4 Austin Shafer 2020-03-18 17:43:26 UTC
Created attachment 212502 [details]
mesa-dri: patch for anv_gem.c

Good ideas, I stole them all and updated the patch.

This works for both mesa-dev and the 18.3.2_9 mesa-dri/mesa-libs in ports.
Comment 5 Austin Shafer 2020-04-01 17:15:56 UTC
removing needs-qa, verified portlint didn't complain about this.

Anything else I need to provide?
Comment 6 commit-hook freebsd_committer freebsd_triage 2020-11-26 20:22:04 UTC
A commit references this bug:

Author: jbeich
Date: Thu Nov 26 20:21:31 UTC 2020
New revision: 556348
URL: https://svnweb.freebsd.org/changeset/ports/556348

Log:
  graphics/mesa-devel: work around userptr for anv on gen < 8 and iris compute

  Vulkan on Intel Broadwell or later uses softpin but older need userptr:

     /* We only allow 48-bit addresses with softpin because knowing the actual
      * address is required for the vertex cache flush workaround.
      */
     device->supports_48bit_addresses = (device->info.gen >= 8) &&
                                        device->has_softpin &&
                                        device->gtt_size > (4ULL << 30 /* GiB */);

     device->use_softpin = device->has_softpin &&
                           device->supports_48bit_addresses;

  PR:		244877
  Submitted by:	Austin Shafer <ashafer@badland.io>

Changes:
  head/graphics/mesa-devel/files/patch-userptr
Comment 7 Jan Beich freebsd_committer freebsd_triage 2021-06-28 20:17:49 UTC
Vulkan on Haswell is still broken in mesa-dri. mesa-devel/files/patch-userptr is a workaround for missing kernel driver feature thus won't be upstreamed.
Comment 8 Jan Beich freebsd_committer freebsd_triage 2021-06-28 20:39:38 UTC
(In reply to Jan Beich from comment #7)
> won't be upstreamed.

... by me. Tracking consumers for the missing feature via downstream-only patch makes it easy to adjust/remove once implemented properly.

Sorry, I forgot what I said in comment 1 but can't remove due to bug 191677.
Comment 9 Jan Beich freebsd_committer freebsd_triage 2021-06-30 04:11:07 UTC
Likely closed by mistake.
Comment 10 Jan Beich freebsd_committer freebsd_triage 2022-06-23 16:14:45 UTC
Upstream Mesa didn't like the workaround, see https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13162
Comment 11 Ivan 2022-07-25 10:26:49 UTC
Hello,

I've tried to use hardware accelerated opencl (with lang/intel-compute-runtime) in graphics/darktable and discovered this PR after receiving 


0.068381 [opencl_init] opencl library 'libOpenCL' found on your system and loaded
ioctl(I915_GEM_USERPTR) failed. Try running as root but expect poor stability.
Abort was called at 115 line in file:
/wrkdirs/usr/ports/lang/intel-compute-runtime/work-llvm13/compute-runtime-22.24.23453/shared/source/os_interface/linux/drm_memory_manager.cpp

I've found discussion here https://github.com/FreeBSDDesktop/kms-drm/issues/197 but still fail to understand what is needed to workaround the issue.

Can you enlighten me, should I apply this patch or try something else?

Workstation uses Skylake CPU, which is rather modern (compared to Broadwell), so I suppose it shouldn't use userptr at all, isn't it?
Comment 12 Jan Beich freebsd_committer freebsd_triage 2022-07-26 01:18:23 UTC
(In reply to Ivan from comment #11)
> ioctl(I915_GEM_USERPTR) failed. Try running as root but expect poor stability.

Did you follow this advice (run the app as root)? The workaround in lang/intel-compute-runtime/files/patch-userptr implements a fallback via a feature that requires root. Without it running as root would have made no difference i.e., still failed.

There's no workaround to avoid root. drm-kmod needs to implement CONFIG_MMU_NOTIFIER but that would be a "fix", not a "workaround".

> Can you enlighten me, should I apply this patch or try something else?

No. attachment 212502 [details] is for Vulkan on Haswell, not OpenCL. Besides, intel-compute-runtime doesn't support Haswell.

> Workstation uses Skylake CPU, which is rather modern (compared to Broadwell),
> so I suppose it shouldn't use userptr at all, isn't it?

Applies to Vulkan except VK_EXT_external_memory_host. OpenCL in Intel NEO (compute-runtime) requires userptr, unlike Beignet where userptr was optional for OpenCL 2.0.