LLVM is a huge beast. Of the 353 ports installed on my VM (running X11), llvm80 amounts to 45% of the total disk space used in /usr/local - over 800M. However I don't need it as I don't run any accelerated X. Can we add a (close to) trivial option that avoids RUN_DEPENDS and BUILD_DEPENDS on devel/llvm80 and just does CONFIGURE_ARGS+=--disable-llvm? That would also dramatically reduce compile time for this simplified case.
What features does llvm provide/enable? Is it *only* for 'accelerated' X ?
(In reply to Kubilay Kocak from comment #1) LLVM is required only for: GALLIUM_DRIVERS=SRWAST GALLIUM_DRIVERS=R300 GALLIUM_DRIVERS=RADEONSI VULKAN_DRIVERS=RADEON It's not needed for anything else. Broadly speaking, if you don't have a Radeon card or don't need acceleration, it's just bloat. (Besides I'm kind of puzzled why we don't simply use the base system's LLVM which for 12-STABLE and above is already LLVM80. Maybe just my lack of understanding.)
(In reply to Helge Oldach from comment #2) > It's not needed for anything else. Broadly speaking, if you don't have a > Radeon card or don't need acceleration, it's just bloat. > LLVM is actually needed for *non-accelerated* graphics, particularly with LLVMpipe. Many desktop environments/toolkits expect some sort of acceleration anymore (even if you don't think it does), and LLVMpipe and the like provide that software rendering. > (Besides I'm kind of puzzled why we don't simply use the base system's LLVM > which for 12-STABLE and above is already LLVM80. Maybe just my lack of > understanding.) -RELEASEs don't change the toolchain originally available upon release, so for 12.0-RELEASE, LLVM is fixed at 6.0.0. Besides, the base system LLVM is built as static libraries and binaries, and some LLVM components not needed/useful to build the base system are stripped away. Think, the base system uses a compiler toolchain *based on LLVM*, in the same way that the base system GCC isn't vanilla either. I won't hesitate to post this wiki page again: https://wiki.freebsd.org/WhyDoIHaveToBuildLLVMWhenIAlreadyHaveClangInstalled
Hi! Adding more options and complexity to mesa is not desirable. It is a big enough beast as is. Furthermore, as Charlie Li points out in Comment #3, llvm is used for non-accelerated graphics as well. The reason we use llvm from ports is two fold. First off, it gives us one unified version of llvm+mesa across all of FreeBSD, which lessens the diversity and makes the port easier to test and maintain. Secondly, and perhaps more importantly, as already been pointed out, the libraries that mesa needs are not provided by base llvm. We understand that this can be resource intensive for some people, but for the reasons listed above, there is no good way around it. Regards Niclas FreeBSD Graphics Team
(In reply to Charlie Li from comment #3) > > It's not needed for anything else. Broadly speaking, if you don't have a > > Radeon card or don't need acceleration, it's just bloat. > > > > LLVM is actually needed for *non-accelerated* graphics, particularly with > LLVMpipe. Not quite, it's needed with SWRAST only. If SWRAST is not compiled in, LLVM is obsolete. As described above.
(In reply to Helge Oldach from comment #5) It's also needed by radeonsi and it's used if you're a guest in virtual machines.
(In reply to Niclas Zeising from comment #4) > Adding more options and complexity to mesa is not desirable. Actually this option was in place until a few days ago! It was called MESA_LLVM_VER. Setting this Makefile variable to an empty string had effectively suppressed any LLVM dependency. Please cross-check with the previous Makefile and Makefile.common before the recent update. And yes, Mesa has indeed been working fine in the non accelerated case. I understand the background of the recent build unification for other platforms, however for the common amd64 and i386 cases this took away a nice feature. > The reason we use llvm from ports is two fold. First off, it gives us > one unified version of llvm+mesa across all of FreeBSD, which lessens > the diversity and makes the port easier to test and maintain. Secondly, > and perhaps more importantly, as already been pointed out, the libraries > that mesa needs are not provided by base llvm. Story is clear, but the price is extremely high. In my case it has almost doubled the disk size of port installation. Also compilation is a burden - LLVM takes hours to build on less capable platforms - it's an order of magnitude more than just Mesa alone! Furthermore this price is paid even when it's not needed - for example in the not too unusual case of a stock Intel CPU with built-in graphics. > We understand that this can be resource intensive for some people It's not the people, it basically disallows using 3d graphics on less capable platforms, and even on decent platforms it's a burden. This really should be made more lightweight.
(In reply to Johannes Lundberg from comment #6) > It's also needed by radeonsi True. I had mentioned the four cases above. > it's used if you're a guest in virtual machines. I can't follow. Virtualbox here. Mesa / X11 running fine without LLVM. Can you explain what I am missing?
(In reply to Helge Oldach from comment #8) LLVMPipe is used in the new DRM based graphics drivers for virtual machines. They're still WIP but coming soon and will improve performance and usability of FreeBSD in VMs. As for asking for special treatment, there's nothing stopping you from keeping your own branch or set of local patches. We all do for stuff we don't want to/shouldn't upstream. What should be done on the other hand, is have poudriere or ports have the ability to install build dependencies (like LLVM) from pre-built packages instead of building from source.
(In reply to Helge Oldach from comment #7) > (In reply to Niclas Zeising from comment #4) > > Adding more options and complexity to mesa is not desirable. > > Actually this option was in place until a few days ago! It was called > MESA_LLVM_VER. Setting this Makefile variable to an empty string had > effectively suppressed any LLVM dependency. Please cross-check with the > previous Makefile and Makefile.common before the recent update. And yes, > Mesa has indeed been working fine in the non accelerated case. This was never supported, and the fact that it did work was an accident. > > I understand the background of the recent build unification for other > platforms, however for the common amd64 and i386 cases this took away a nice > feature. > > > The reason we use llvm from ports is two fold. First off, it gives us > > one unified version of llvm+mesa across all of FreeBSD, which lessens > > the diversity and makes the port easier to test and maintain. Secondly, > > and perhaps more importantly, as already been pointed out, the libraries > > that mesa needs are not provided by base llvm. > > Story is clear, but the price is extremely high. In my case it has almost > doubled the disk size of port installation. Also compilation is a burden - > LLVM takes hours to build on less capable platforms - it's an order of > magnitude more than just Mesa alone! > > Furthermore this price is paid even when it's not needed - for example in > the not too unusual case of a stock Intel CPU with built-in graphics. > > > We understand that this can be resource intensive for some people > > It's not the people, it basically disallows using 3d graphics on less > capable platforms, and even on decent platforms it's a burden. This really > should be made more lightweight. It is possible to install both llvm and mesa from pkg, they work quite well, and you don't have to build everything yourself. Changing the mesa ports to make llvm optional, even if possible, makes these already quite complex ports even more complex, and adds further use cases to test for. All this requires more man power. Regards Niclas Zeising FreeBSD Graphics Team
(In reply to Johannes Lundberg from comment #9) > What should be done on the other hand, is have poudriere or ports > have the ability to install build dependencies (like LLVM) from pre-built > packages instead of building from source. For poudriere see https://github.com/freebsd/poudriere/issues/319 For ports someone should teach USE_PACKAGE_DEPENDS/USE_PACKAGE_DEPENDS_ONLY to try "pkg install", not just "pkg add".
Created attachment 205492 [details] patch adding a LLVM configure knob
(In reply to Niclas Zeising from comment #10) > Changing the mesa ports to make llvm optional, even if possible, makes these > already quite complex ports even more complex, and adds further use cases to > test for. All this requires more man power. Glad to help out with a quick patch. Note this actually simplifies the --enable-llvm/--disable-llvm logic.
(In reply to Helge Oldach from comment #13) > (In reply to Niclas Zeising from comment #10) > > Changing the mesa ports to make llvm optional, even if possible, makes these > already quite complex ports even more complex, and adds further use cases to > test for. All this requires more man power. > > Glad to help out with a quick patch. Note this actually simplifies the > --enable-llvm/--disable-llvm logic. Hi! Thank you for the patch. Unfortunately, we won't be able to use it. LLVM is a hard dependency for the radeonsi and amdgpu drivers. It's also required for llvmpipe, which is the default software renderer and used by all desktops without a GPU. Working through all the dependencies for a mesa port without LLVM would be quite complex. Graphics is a complex area and to provide the best possible experience for the vast majority of our users, we want to ensure all the options we offer work. Making LLVM optional would create a testing matrix that is beyond our ability to even spot check, let alone ensure works for the majority of all our users. It would be an ongoing burden to the team with each new mesa release, as which drivers use what bits of LLVM are constantly changing. Taking all these factors into account, we're simple unable to support making this optional at this time. We do not have the manpower for it. Of course, you are welcome to continue using your patch, for as long as it works, but you'll have to work out any issues that are caused by it. Regards Niclas Zeising FreeBSD Graphics Team
After reading this PR, I'm still interested in build with base compiler. On rip4/aarch64 the only dep on ports llvm is: % pkg info -r llvm19 llvm19-19.1.7_1: mesa-dri-24.1.7_4 What the best way to quick test mesa-dri with base compiler? It will save me more than 15 hours building llvm19 just for this. Thanks
(In reply to Nuno Teixeira from comment #15) The system's clang/clang++ build (from based on LLVM) does not include what mesa-dri uses from LLVM.
(In reply to Nuno Teixeira from comment #15) In fact, mesa-dri uses LLVM as a library, not as a compiler (not clang/clang++): USES+= llvm:lib,noexport mesa-dri also uses other ports that are LLVM based: devel/libclc@${LLVM_PORT:T} spirv-llvm-translator@${LLVM_PORT:T} Those references are from doing: # grep -i llvm /usr/ports/graphics/mesa-dri/Makefile USES+= llvm:lib,noexport iris_BUILD_DEPENDS= libclc-${LLVM_PORT:T}>0:devel/libclc@${LLVM_PORT:T} \ spirv-llvm-translator-${LLVM_PORT:T}>0:devel/spirv-llvm-translator@${LLVM_PORT:T} spirv-llvm-translator-${LLVM_PORT:T}>0:devel/spirv-llvm-translator@${LLVM_PORT:T} iris_CONFIGURE_ENV= PKG_CONFIG_PATH="${LLVM_PREFIX}/libdata/pkgconfig" libclc-${LLVM_PORT:T}>0:devel/libclc@${LLVM_PORT:T} \ spirv-llvm-translator-${LLVM_PORT:T}>0:devel/spirv-llvm-translator@${LLVM_PORT:T} spirv-llvm-translator-${LLVM_PORT:T}>0:devel/spirv-llvm-translator@${LLVM_PORT:T} anv_CONFIGURE_ENV= PKG_CONFIG_PATH="${LLVM_PREFIX}/libdata/pkgconfig"
(In reply to Mark Millard from comment #17) Ok, but meson options have: option( 'llvm', type : 'feature', deprecated: {'true': 'enabled', 'false': 'disabled'}, description : 'Build with LLVM support.' ) option( 'shared-llvm', type : 'feature', deprecated: {'true': 'enabled', 'false': 'disabled'}, description : 'Whether to link LLVM shared or statically.' ) option( 'draw-use-llvm', type : 'boolean', value : true, description : 'Whether to use LLVM for the Gallium draw module, if LLVM ' + 'is included.' ) And maybe there is a possibility to build it without llvm dep for systems/arches that doesn't need it?
(In reply to Nuno Teixeira from comment #18) (...) Like #c2 mentions: LLVM is required only for: GALLIUM_DRIVERS=SRWAST GALLIUM_DRIVERS=R300 GALLIUM_DRIVERS=RADEONSI VULKAN_DRIVERS=RADEON
(In reply to Nuno Teixeira from comment #19) None of which does the system have llvm library materials for supporting mesa-dri's use of llvm libraries (not clang/clang++). You still have to deal with a ports llvm to get the llvm library(s) involved that mesa-dri uses.
(In reply to Nuno Teixeira from comment #19) Needing LLVM here, is not the same as needing clang/clang++ . The system only builds the clang/clang++ related material from LLVM, not the rest of it --not what mesa-dri uses. Being able to build a minimal mesa-dri for a type of context and being able to build a for-that-mesa-dri-only LLVM (no clang or clang++) would be a better fit to small arm boards, like the RPi4. My point overall here was just that comemnt #15's "base compiler" reference is not a valid direction based on what the system builds vs. not from what all LLVM allows to be built. mesa-dri is not trying to use clang/clang++ but is instead using library(s) from LLVM. Outside of that, the lists are probably a better place for discussion of the overall problem for your context, not here. I've sent a reply to your freebsd-arm list note that referenced here.
(In reply to Nuno Teixeira from comment #19) I suggest you move to the list and send out a note on which mesa-dri options you actually need. From that a minimal mesa-dri build could probably be figured out, even if it required a modified Makefile. It may or may not end up needing any LLVM libraires. So LLVM might end up not involved on the basis of the minimal mesa-dri option set that you need. If LLVM libraries did end up involved, a minimal build that avoids building any clang/clang++ may be a option, even if it requires modifying the Makefile.
(In reply to Mark Millard from comment #22) Done, I've just sent to list a build with (almost) all options disabled. Note: that list is what is presented on aarch64 arch. After I have some run tests on that machine: x11/i3/firefox, we will have more clues about llvm deps and more info to make a minimal mesa-dri port. Thanks
Keep in mind mesa-dri wants mesa-libs which has USES=llvm:noexport. For my specific graphic card model I have only X11 and swrast enabled and everything else off (for mesa-libs and mesa-dri). Since this PR got closed 5 years ago, I've been fine with below tweak. I think it would suite the RPi4/aarch64 case likewise? diff --git a/graphics/mesa-dri/Makefile b/graphics/mesa-dri/Makefile index be54794d3a73..f60286a54d24 100644 --- a/graphics/mesa-dri/Makefile +++ b/graphics/mesa-dri/Makefile @@ -10,8 +10,6 @@ BUILD_DEPENDS+= ${PYTHON_PKGNAMEPREFIX}ply>0:devel/py-ply@${PY_FLAVOR} LIB_DEPENDS+= libglapi.so:graphics/mesa-libs -USES+= llvm:lib,noexport - OPTIONS_DEFINE= ZSTD OPTIONS_GROUP= GALLIUM PLATFORM VULKAN OPTIONS_DEFAULT= WAYLAND X11 ZSTD diff --git a/graphics/mesa-dri/Makefile.common b/graphics/mesa-dri/Makefile.common index a97e27012721..2e7dafb0835c 100644 --- a/graphics/mesa-dri/Makefile.common +++ b/graphics/mesa-dri/Makefile.common @@ -77,5 +77,5 @@ MESON_ARGS+= -Dlibelf=disabled LDFLAGS+= -Wl,--undefined-version LDFLAGS+= -Wl,-rpath=${LOCALBASE}/llvm${LLVM_VERSION}/lib -MESON_ARGS+= -Dllvm=enabled \ +MESON_ARGS+= -Dllvm=disabled \ -Dlibunwind=disabled diff --git a/graphics/mesa-libs/Makefile b/graphics/mesa-libs/Makefile index 7ffe5201b2a5..4c99f446a048 100644 --- a/graphics/mesa-libs/Makefile +++ b/graphics/mesa-libs/Makefile @@ -9,8 +9,6 @@ WWW= https://www.mesa3d.org/ LICENSE= MIT LICENSE_FILE= ${WRKSRC}/docs/license.rst -USES= llvm:noexport - BUILD_DEPENDS= libglvnd>=0:graphics/libglvnd OPTIONS_GROUP= PLATFORM
(In reply to Helge Oldach from comment #24) Nice, that's exactly what I'm looking for! I'm discussing this with Mark Milard at lists and there is a doubt of whether software accel will be used on rpi4 (driver scfb framebuffer). You are using swrast. I will test laters if it makes a difference on rpi4. Thanks!
(In reply to Nuno Teixeira from comment #19) [This goes along with Helge's notes.] I'd done a bulk -a build experiment (that generally cause llvm20 to be used). The mesa-libs log reports: ===> Installing existing package /packages/All/llvm20-20.1.0.r1.pkg [aarch64PBase] Installing llvm20-20.1.0.r1... [aarch64PBase] `-- Installing libedit-3.1.20240808,1... [aarch64PBase] `-- Extracting libedit-3.1.20240808,1: .......... done [aarch64PBase] `-- Installing lua53-5.3.6_1... [aarch64PBase] `-- Extracting lua53-5.3.6_1: .......... done [aarch64PBase] `-- Installing perl5-5.36.3_2... [aarch64PBase] `-- Extracting perl5-5.36.3_2: .......... done [aarch64PBase] `-- Installing zstd-1.5.6... [aarch64PBase] | `-- Installing liblz4-1.10.0,1... [aarch64PBase] | `-- Extracting liblz4-1.10.0,1: .......... done [aarch64PBase] `-- Extracting zstd-1.5.6: .......... done [aarch64PBase] Extracting llvm20-20.1.0.r1: .......... done And the mesa-dri log shows, not only llvm20 directly, but: ===> Installing existing package /packages/All/mesa-libs-24.1.7_1.pkg [aarch64PBase] Installing mesa-libs-24.1.7_1... [aarch64PBase] `-- Installing expat-2.6.4... [aarch64PBase] `-- Extracting expat-2.6.4: .......... done [aarch64PBase] `-- Installing libXxf86vm-1.1.5... [aarch64PBase] `-- Extracting libXxf86vm-1.1.5: .......... done [aarch64PBase] `-- Installing libdrm-2.4.123,1... [aarch64PBase] | `-- Installing libpciaccess-0.18.1_1... [aarch64PBase] | | `-- Installing hwdata-0.392,1... [aarch64PBase] | | `-- Extracting hwdata-0.392,1: .......... done [aarch64PBase] | `-- Extracting libpciaccess-0.18.1_1: ........ done [aarch64PBase] `-- Extracting libdrm-2.4.123,1: .......... done [aarch64PBase] `-- Installing wayland-1.23.1... [aarch64PBase] | `-- Installing libepoll-shim-0.0.20240608... [aarch64PBase] | `-- Extracting libepoll-shim-0.0.20240608: .......... done [aarch64PBase] | `-- Installing libxml2-2.11.9... [aarch64PBase] | `-- Extracting libxml2-2.11.9: .......... done [aarch64PBase] `-- Extracting wayland-1.23.1: .......... done [aarch64PBase] `-- Installing zstd-1.5.6... [aarch64PBase] | `-- Installing liblz4-1.10.0,1... [aarch64PBase] | `-- Extracting liblz4-1.10.0,1: .......... done [aarch64PBase] `-- Extracting zstd-1.5.6: .......... done [aarch64PBase] Extracting mesa-libs-24.1.7_1: .......... done But the report you got for what used (your equivalent of) llvm20 here, did not show mesa-libs at all. Your means of listing what uses the llvm that you have been building has not been showing you build-time dependencies that do not end up with run-time dependencies. But the build-time dependencies, if not removed, still lead to needing to have built the llvm in question (with sufficient options for all the actual usage to be supported).