Bug 230789

Summary: graphics/mesa-dri: switch to devel/llvm80
Product: Ports & Packages Reporter: Jan Beich <jbeich>
Component: Individual Port(s)Assignee: freebsd-x11 (Nobody) <x11>
Status: Closed FIXED    
Severity: Affects Only Me CC: arrowd, brooks, cpm, jcfyecrayz, maciej, rkoberman, rozhuk.im, vvd, x11, zeising
Priority: --- Keywords: patch
Version: LatestFlags: zeising: maintainer-feedback-
Hardware: Any   
OS: Any   
See Also: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=233652
Bug Depends on: 227682    
Bug Blocks: 235570    
Attachments:
Description Flags
For Mesa 18.1
none
v1
none
v2
jbeich: maintainer-approval? (x11)
beignet + llvm80 none

Description Jan Beich freebsd_committer freebsd_triage 2018-08-20 23:07:53 UTC
Created attachment 196405 [details]
For Mesa 18.1
Comment 1 Jan Beich freebsd_committer freebsd_triage 2018-08-20 23:34:38 UTC
Created attachment 196406 [details]
v1

Oops, drop patch that will conflict with bug 230298.
Comment 2 Niclas Zeising freebsd_committer freebsd_triage 2018-08-24 14:37:43 UTC
Hi!
llvm70 is still in RC in the ports tree.  I would like to see it get bumped to a release version before this goes in.
I also hope that gecko ports and other big consumers of ports llvm70 gets bumped at the same time.  That way there won't be a window where you would need two llvm versions installed.
Comment 3 Brooks Davis freebsd_committer freebsd_triage 2018-08-24 15:56:06 UTC
FYI, llvm70 is definitely currently broken on i386 (it generates lib calls for atomics what we don't implement).  dim@ is working on getting the change that caused this fixed.
Comment 4 Jan Beich freebsd_committer freebsd_triage 2018-08-26 17:57:19 UTC
(In reply to Niclas Zeising from comment #2)
> I also hope that gecko ports and other big consumers of ports llvm70
> gets bumped at the same time.  That way there won't be a window
> where you would need two llvm versions installed.

I'd like to start somewhere.
- bug 230790 will bump 10.4/11.1 users to Clang 7.0 but after 10.4/11.1 EOL that change would be nop.
- gecko@ needs clang-sys to support "clang_7_0" bindings or bindgen may crash when building Stylo but, surprisingly, it doesn't, see https://ptpb.pw/mpif

(In reply to Brooks Davis from comment #3)
Mesa doesn't use LLVM for atomics, so the change here builds fine on all Clang architectures. For one, llvmpipe (installed as swrast_dri.so) works fine on 10.4 i386 (tested Firefox 63 with WebRender enabled).
Comment 5 Jan Beich freebsd_committer freebsd_triage 2018-09-23 17:47:15 UTC
I plan to land this shortly after 2018Q4 branches. Mesa itself needs newer LLVM in order to support recent (mainly AMD) hardware better.

llvm60 is unlikely to receive upstream backports now that llvm70 is stable. Projects that support more than one stable branch are actually rare due to tremendous waste of resources. Even Mesa quickly sunsets previous branch after X.Y.1 release.
Comment 6 Niclas Zeising freebsd_committer freebsd_triage 2018-09-24 15:33:52 UTC
As I've stated before, I'd like for this to happen to most of the ports tree at once.  Only bumping mesa just means that everyone will need to have two versions of llvm installed in order to get other desktop applications, such as gecko ports.
Comment 7 Tobias Kortkamp freebsd_committer freebsd_triage 2018-10-14 08:46:42 UTC
(In reply to Niclas Zeising from comment #6)
> As I've stated before, I'd like for this to happen to most of the
> ports tree at once.

What's the plan to make this happen?

> Only bumping mesa just means that everyone will need to have two
> versions of llvm installed in order to get other desktop applications,
> such as gecko ports.

Since ports r481071 Gecko ports require llvm70 to build as well.
Comment 8 Tobias Kortkamp freebsd_committer freebsd_triage 2018-11-16 07:04:04 UTC
Ping?  Another month has passed.

Can we be clued into the roadmap to llvm70 here?
Comment 9 Niclas Zeising freebsd_committer freebsd_triage 2018-11-16 08:10:12 UTC
(In reply to Tobias Kortkamp from comment #8)

We recently switched wayland default to on for mesa ports.  I'd like for that to settle before switching this as well.  We have a graphics team meeting next week, after that.
Comment 10 Jan Beich freebsd_committer freebsd_triage 2018-11-29 13:23:35 UTC
Created attachment 199665 [details]
v2

Hop onto DEFAULT_VERSIONS train after ports r485466. Obviously, if x11@ is gonna drag progress during the next bump Mesa would return to the local default.
Comment 11 Tobias Kortkamp freebsd_committer freebsd_triage 2018-11-29 13:31:14 UTC
(In reply to Niclas Zeising from comment #9)
> We have a graphics team meeting next week, after that.

Are you still planning on publishing meeting notes like in October?
Comment 12 Niclas Zeising freebsd_committer freebsd_triage 2018-11-29 14:38:15 UTC
(In reply to Tobias Kortkamp from comment #11)
> Are you still planning on publishing meeting notes like in October?

Yes, that's the plan.  I'm looking into why this hasn't happened yet.
Comment 13 Jan Beich freebsd_committer freebsd_triage 2019-01-26 12:38:05 UTC
After bug 235215 lands v2 will switch mesa-* to llvm80. Pick v1 if you want to stay on llvm70.
Comment 14 Tobias Kortkamp freebsd_committer freebsd_triage 2019-01-28 14:51:34 UTC
mesa-dri-18.3.2 with llvm80-8.0.0.r1 and amdgpu needs a backport
of [1] to avoid

 '+vgpr-spilling' is not a recognized feature for this target (ignoring feature)

errors.

[1] https://gitlab.freedesktop.org/mesa/mesa/commit/9cab8ccd6cb5aa8a4748dd6d7fc15d9747624df6
Comment 15 Jan Beich freebsd_committer freebsd_triage 2019-01-30 13:33:09 UTC
Created attachment 201531 [details]
beignet + llvm80

(After v2) lang/beignet fails with DEFAULT_VERSIONS+=llvm=80, so here's a fix. Tested on Skylake GT2 via graphics/waifu2x-converter-cpp.

Rationale is in the patch header. I suspect Debian will send a similar fix upstream but given bug 233652 there's not much hope.
Comment 16 commit-hook freebsd_committer freebsd_triage 2019-03-04 10:43:23 UTC
A commit references this bug:

Author: jbeich
Date: Mon Mar  4 10:42:24 UTC 2019
New revision: 494579
URL: https://svnweb.freebsd.org/changeset/ports/494579

Log:
  graphics/mesa-dri: switch to llvm70

  PR:		230789
  Approved by:	maintainer timeout (2 weeks)

Changes:
  head/devel/libclc/Makefile
  head/graphics/libosmesa/Makefile
  head/graphics/mesa-dri/Makefile
  head/graphics/mesa-dri/Makefile.common
  head/lang/beignet/Makefile
  head/lang/clover/Makefile
Comment 17 commit-hook freebsd_committer freebsd_triage 2019-03-04 10:56:39 UTC
A commit references this bug:

Author: jbeich
Date: Mon Mar  4 10:56:14 UTC 2019
New revision: 494583
URL: https://svnweb.freebsd.org/changeset/ports/494583

Log:
  graphics/mesa-dri: back out r494579

  Landed by mistake. x11@ is immune to maintainer timeout nowadays.

  PR:		230789

Changes:
  head/devel/libclc/Makefile
  head/graphics/libosmesa/Makefile
  head/graphics/mesa-dri/Makefile
  head/graphics/mesa-dri/Makefile.common
  head/lang/beignet/Makefile
  head/lang/clover/Makefile
Comment 18 Jan Beich freebsd_committer freebsd_triage 2019-03-20 10:35:35 UTC
As mesa-* continues to drag updates jumping on LLVM_DEFAULT may become harder in future. The plan is to bump LLVM_DEFAULT to 90 around October.
Comment 19 Vladimir Druzenko freebsd_committer freebsd_triage 2019-03-21 00:47:15 UTC
(In reply to Jan Beich from comment #15)
This patch work for me to build beignet with llvm80.
You can commit it before decided to remove beignet or not.
Comment 20 Tobias Kortkamp freebsd_committer freebsd_triage 2019-04-02 17:03:46 UTC
Any updates here?  I do not see anything related to moving Mesa to a newer
LLVM in recent graphics team meeting notes (latest mention in 2019-01-30 notes,
after that nothing).
Comment 21 commit-hook freebsd_committer freebsd_triage 2019-06-30 14:56:29 UTC
A commit references this bug:

Author: zeising
Date: Sun Jun 30 14:56:14 UTC 2019
New revision: 505425
URL: https://svnweb.freebsd.org/changeset/ports/505425

Log:
  Switch mesa and related ports to llvm80

  Switch mesa over to use llvm80 instead of llvm60.  Make it use the
  global LLVM_DEFAULT instead of deciding for ourself which llvm version
  to use. [1]

  Fix build of lang/beginet [1]

  Add patch from upstream to fix build of devel/libclc.  The patch is taken
  from the git mirror of devel/libclc rather than the SVN repo, for
  convenience.

  Add a patch from mesa upstream preventing certain error messages when using
  amdgpu [2]

  Add a notice to bsd.default-versions.mk asking that the graphics team be
  informed before the llvm version is changed.

  Enable llvm and gallium on MIPS.  As far as I can tell, this used to
  be the default before this change.

  Bump portrevisions since dependencies changed.

  PR:		230789 [1], [2]
  Submitted by:	jbeich [1], tobik [2]
  Obtained from:	FreeBSD Graphics Team development repo
  		https://github.com/FreeBSDDesktop/freebsd-ports/commits/feature/mesa-llvm80
  Sponsored by:	B3 Init (zeising)

Changes:
  head/Mk/bsd.default-versions.mk
  head/devel/libclc/Makefile
  head/devel/libclc/files/patch-62a9191.c
  head/graphics/libosmesa/Makefile
  head/graphics/mesa-dri/Makefile
  head/graphics/mesa-dri/Makefile.common
  head/graphics/mesa-dri/files/patch-9cab8cc.c
  head/graphics/mesa-libs/Makefile
  head/lang/beignet/Makefile
  head/lang/beignet/files/patch-llvm8
  head/lang/clover/Makefile
Comment 22 Niclas Zeising freebsd_committer freebsd_triage 2019-06-30 14:57:07 UTC
Committed.  Leaving this open for a bit in case there's fallout.
Comment 23 Vladimir Druzenko freebsd_committer freebsd_triage 2019-06-30 16:45:25 UTC
Just tested build with python 3.6:

--- graphics/mesa-dri/Makefile.common.orig
+++ graphics/mesa-dri/Makefile.common
@@ -45,7 +45,7 @@
 .endif
 
 USES+=         compiler:c++11-lib bison gettext-tools gmake libtool \
-               localbase pathfix pkgconfig python:2.7,build shebangfix tar:xz
+               localbase pathfix pkgconfig python:3.6,build shebangfix tar:xz
 USE_LDCONFIG=  yes
 GNU_CONFIGURE= yes

Build without errors.

May be time to move to new version of the python?…
Comment 24 John Hein 2019-07-02 15:03:36 UTC
One issue:  if llvm80 was not built with the BE_STANDARD option (e.g., BE_FREEBSD instead), then you get this error in graphics/mesa-dri at configure time:

 .
 .
checking for XLIB_RANDR... yes
checking for EXPAT... yes
checking for RADEON... yes
checking for RADEON... yes
checking for I915... yes
checking for AMDGPU... yes
configure: error: LLVM target 'amdgpu' not enabled in your LLVM build. Required by radv.
===>  Script "configure" failed unexpectedly.


The fix is to build llvm80 with BE_STANDARD of course, but the failure is a bit ungraceful.  If nothing else, it's documented here for the unaware to find.
Comment 25 Brooks Davis freebsd_committer freebsd_triage 2019-07-02 16:31:41 UTC
I can add amdgpu to BE_FREEBSD if this is likely to be an issue.
Comment 26 Niclas Zeising freebsd_committer freebsd_triage 2019-07-02 18:21:05 UTC
(In reply to John Hein from comment #24)
> One issue:  if llvm80 was not built with the BE_STANDARD option (e.g.,
> BE_FREEBSD instead), then you get this error in graphics/mesa-dri at
> configure time:
> 

[snip error

> 
> The fix is to build llvm80 with BE_STANDARD of course, but the failure is a
> bit ungraceful.  If nothing else, it's documented here for the unaware to
> find.

Interesting.  I have to think for a second if it's possible to detect if this is enabled in llvm/clang before starting the build.  Otherwise, perhaps enabling it for BE_FREEBSD, as brooks@ suggest in comment #25 might be the best option.
Comment 27 John Hein 2019-07-02 20:35:33 UTC
FYI, 'llvm-config80 --targets-built' is how mesa-dri's configure tries to detect if AMDGPU is supported.
Comment 28 rkoberman 2019-08-05 18:45:42 UTC
Any change in the status of this? I really hate having to build all the standard backends. Just building with BE_FREEBSD takes a very long time. (No, I have not compared.)
Comment 29 Niclas Zeising freebsd_committer freebsd_triage 2019-08-05 19:05:21 UTC
(In reply to rkoberman from comment #28)
> Any change in the status of this? I really hate having to build all the
> standard backends. Just building with BE_FREEBSD takes a very long time.
> (No, I have not compared.)

You probably have to talk to the llvm maintainer about always enabling the AMDGPU backend.  Trying to get mesa to only build amdgpu if AMDGPU is available in llvm will probably be tricky, although I am willing to consider it if you can help out with providing patches.
Comment 30 Brooks Davis freebsd_committer freebsd_triage 2019-08-05 20:41:23 UTC
Is AMDGPU the only backed used by mesa?
Comment 31 Jan Beich freebsd_committer freebsd_triage 2019-08-05 21:48:57 UTC
(In reply to Brooks Davis from comment #30)
> Is AMDGPU the only backed used by mesa?

See https://gitlab.freedesktop.org/mesa/mesa/blob/mesa-18.3.2/meson.build#L1173-1187 For one, with_gallium_opencl is currently part of lang/clover.
Comment 32 commit-hook freebsd_committer freebsd_triage 2019-08-07 19:54:03 UTC
A commit references this bug:

Author: brooks
Date: Wed Aug  7 19:53:27 UTC 2019
New revision: 508349
URL: https://svnweb.freebsd.org/changeset/ports/508349

Log:
  Assorted minor improvements:

  - Add a build conflict for commonmark-cmark-* when DOCS are enabled.
    This prevents a failure later on in the build. [0]

  - Add a new option BE_AMDGPU which can be used to enable the AMDGPU
    backed used by mesa when BE_NATIVE or BE_FREEBSD is set.  Enable this
    option by default to limit later surprises. [1]

  - Use LLVM_ENABLE_Z3_SOLVER instead of the now removed
    CLANG_ANALYZER_ENABLE_Z3_SOLVER to disable Z3 discovery and linkage.

  PR:		239636 [0], 230789 [1]

Changes:
  head/devel/llvm90/Makefile
Comment 33 Vladimir Druzenko freebsd_committer freebsd_triage 2019-08-18 01:34:00 UTC
(In reply to VVD from comment #23)
--- graphics/mesa-dri/Makefile.common.orig
+++ graphics/mesa-dri/Makefile.common
@@ -45,7 +45,7 @@
 .endif
 
 USES+=         compiler:c++11-lib bison gettext-tools gmake libtool \
-               localbase pathfix pkgconfig python:2.7,build shebangfix tar:xz
+               localbase pathfix pkgconfig python:build shebangfix tar:xz
 USE_LDCONFIG=  yes
 GNU_CONFIGURE= yes

Work fine with python 3.6 and 3.7 on 12.0 amd64 and i386.
Comment 34 Niclas Zeising freebsd_committer freebsd_triage 2019-08-18 06:52:56 UTC
(In reply to VVD from comment #33)

Can you open a separate bug to track this?
Comment 35 Vladimir Druzenko freebsd_committer freebsd_triage 2019-08-18 13:09:58 UTC
(In reply to Niclas Zeising from comment #34)
Ok: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=239939
Comment 36 commit-hook freebsd_committer freebsd_triage 2019-09-06 11:04:59 UTC
A commit references this bug:

Author: brooks
Date: Fri Sep  6 11:04:45 UTC 2019
New revision: 511300
URL: https://svnweb.freebsd.org/changeset/ports/511300

Log:
  Assorted build improvements:

   - Add a build conflict for commonmark-cmark-* when DOCS are enabled.
     This prevents a failure later on in the build. [0]

   - Add a new option BE_AMDGPU which can be used to enable the AMDGPU
     backed used by mesa when BE_NATIVE or BE_FREEBSD is set.  Enable this
     option by default to limit later surprises. [1]

   - New option PYCLANG to add python binding for clang. [2]

  PR:		239636 [0], 230789 [1], 239990 [2]
  Submitted by:	chardon.frederic@gmail.com [2]
  Sponsored by:	DARPA, AFRL

Changes:
  head/devel/llvm80/Makefile
  head/devel/llvm80/pkg-plist