Bug 171914 - New port: devel/pocl: Portable open source OpenCL 1.2 implementation
Summary: New port: devel/pocl: Portable open source OpenCL 1.2 implementation
Status: Closed FIXED
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: Normal Affects Only Me
Assignee: Koop Mast
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2012-09-24 11:10 UTC by O. Hartmann
Modified: 2016-11-21 14:41 UTC (History)
6 users (show)

See Also:


Attachments
file.shar (9.26 KB, text/plain)
2012-09-24 11:10 UTC, O. Hartmann
no flags Details
*very* rough port update to the recent v0.10 release (5.56 KB, application/x-shar)
2015-01-14 21:49 UTC, Johannes M Dieterich
no flags Details
improved Makefile (5.51 KB, text/plain)
2015-01-14 22:40 UTC, Johannes M Dieterich
no flags Details
POCL 0.11-SHAR (9.54 KB, text/plain)
2015-05-02 09:41 UTC, O. Hartmann
no flags Details
POCL-0.13 released, utilizing OpenCL 2.0 and LLVM 3.8.0 (7.25 KB, text/plain)
2016-04-11 17:12 UTC, O. Hartmann
ohartmann: maintainer-approval+
Details
pocl.diff (new port as diff as recommended by porter's handbook) (10.32 KB, patch)
2016-09-03 19:20 UTC, O. Hartmann
no flags Details | Diff
Current pocl 0.13 without the sysinfo patch (10.19 KB, patch)
2016-10-20 19:08 UTC, Koop Mast
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description O. Hartmann 2012-09-24 11:10:06 UTC
FreeBSD lacks in prefessional OpenCL/GPGPU support. Apart from the lack of suitable driver support from the GPU vendors, OpenCL also provides support for tha CPU as a "native" target.

To provide even FreeBSD users, develeopers and scientists to develop software for OpenCL 1.2, it is necessary to have a working library to start with.

The project "POCL", Portable OpenCL, provides via LLVM backend such a library for at least the "native" CPU target, as their website states:


"Portable OpenCL is an open source implementation of the OpenCL standard which can be easily adapted for new targets. One of the goals of the project is improving performance portability of OpenCL programs, avoiding the need for target-dependent manual optimizations. A "native" target is included, which allows running OpenCL kernels on the host (CPU)."

Source: https://launchpad.net/pocl

Fix: Patch attached with submission follows:
How-To-Repeat: This port has been successfully compiled on a most recent FreeBSD 10.0-CURRENT/amd64 system which has been completely compiled with CLANG 3.2 and LLVM 3.2 installed from ports devel/llvm-devel.

The port also has been successfully compiled and installed on FreeBSD 9.1-PRE/amd64, also completely compiled with the CLANG (v 3.1) aboard and LLVM 3.1 installed from the port devel/llvm.

The usefulness hasn't been approved, nor did I do benchmarks for the "native" CPU target.

I hope this [roject, due to its LLVM backend, could gain more assistance and more attention.
Comment 1 Marcus von Appen freebsd_committer freebsd_triage 2012-12-22 13:23:55 UTC
Responsible Changed
From-To: freebsd-ports-bugs->mva

I'll take it.
Comment 2 Marcus von Appen freebsd_committer freebsd_triage 2013-05-13 19:14:18 UTC
State Changed
From-To: open->feedback

As discussed via mail, version 0.7.0 of pocl seems to be released. Do 
you mind to update the port? Thanks.
Comment 3 Marcus von Appen freebsd_committer freebsd_triage 2013-05-18 09:31:58 UTC
Hi,

On, Mon May 13, 2013, O. Hartmann wrote:

[...]

> Hello,
>
> as it has been published in the POCL mailing list, version 0.8 is on the
> brink and compiling on FreeBSD due to a better support. There are some
> issue I try to handle with my limited knowledge, but I'd like to update
> much more to 0.8 than 0.7.
>
> There is also still a problem in compiling/configuring "ocl-icd", which
> is an external package found at
>
> https://forge.imag.fr/projects/ocl-icd/
>
> which we should have in conjunction with POCL.
>
> Since POCL gets more attention, even by AMD.
>
> I will update the proposed port for 0.8 as it gets released. Is that all
> right?

this is perfectly fine with me. I'll set this on hold once more then.

Cheers
Marcus
Comment 4 Marcus von Appen freebsd_committer freebsd_triage 2013-05-19 14:19:17 UTC
State Changed
From-To: feedback->suspended

Waiting for the 0.8 release
Comment 5 Tijl Coosemans freebsd_committer 2013-10-18 10:09:39 UTC
State Changed
From-To: suspended->open

Feedback received.
Comment 6 Carlo Strub freebsd_committer 2014-09-15 05:12:25 UTC
Is this PR still relevant?
Comment 7 florian.ziesche+freebsd 2014-09-18 03:04:51 UTC
(In reply to Carlo Strub from comment #6)
> Is this PR still relevant?

Yes, it is. This is still the only proper way of getting full OpenCL support on FreeBSD. (freeocl is lacking a lot of functionality and we don't even have to talk about support from Nvidia/Intel/AMD ...)

Also, the last patch from ohartman that was attached here https://www.freebsd.org/cgi/query-pr.cgi?pr=171914 seems to have been lost in the migration? This was for pocl 0.8 and it worked pretty well. I'm wondering why this was never added to ports?
Comment 8 O. Hartmann 2014-09-18 05:32:13 UTC
Due to time constraints and the lack in a deeper knowledge of POCL the ports is slowed down.
My personal framework is at POCL-0.10, but with FreeBSD, there is a serious code generation bug in FreeBSD's LLVM/CLANG 3.4.2 (from ports). This happens not to happen on Core2Duo architecures, but on every CPU architecture > C2D, several tests fail and it seems to be obvious that only FreeBSD has this bug. Linux doesn't. It may be due to my personal setup (I use always some strong optimizations on the compiler).

Upstream is informed, but will be fixed earliest in POCL-0.11.

I have no equipment to build ports automatically for several OS types, like 9.3-STABLE, 10-STABLE etc., so I'm stuck with 11-CURRENT for now. This has been shown as a problem in the past.

I will emmit the new SHAR the next days - along with two ports for ICD (ocl-icd and khronos-icd).

For short: I think it is relevant, but my time is really limited. 

Regards,
Oliver
Comment 9 Carlo Strub freebsd_committer 2014-09-18 08:18:57 UTC
Thanks for your feedback. Let's put it to "open".
Comment 10 Johannes M Dieterich freebsd_committer 2015-01-14 21:49:41 UTC
Created attachment 151645 [details]
*very* rough port update to the recent v0.10 release

A bit late to the party, but I just checked with devel/llvm35 on their trunk (the latest release on their side is 0.10 supposed to work with llvm35 and fixing issue #123 ohartmann opened) and it does compile out of the box with the following configure:

./configure --disable-icd CPPFLAGS="-I/usr/local/include" LDFLAGS="-L/usr/local/lib/" LLVM_CONFIG="/usr/local/llvm35/bin/llvm-config"

With the mesa10.4 development branch now featuring a Clover port, it would make sense IMHO to get this into the tree and also look into resolving the upcoming ICD challenges with the guys responsible there.

Bottom line: I may be able to spare a few cycles on this, how can I help?

Find attached a (probably violating close to a thousand port rules) adjustment of the original port to v0.10. It does however compile...

Please note that I removed the options for gcc vs llvm and hardwired llvm34. I feel this is in line with the Mesa ports. Personally, I may consider offering llvm34 vs llvm35 but that is another story, I suppose.
Comment 11 Johannes M Dieterich freebsd_committer 2015-01-14 22:40:21 UTC
Created attachment 151648 [details]
improved Makefile
Comment 12 O. Hartmann 2015-01-15 07:25:23 UTC
Sorry for the late response.

I already have/had a "compiling" port infrastructure for 0.10, but there was a serious bug with LLVM/CLANG 3.4 only occuring in FreeBSD, not on any Linux platform. 

For the ICD: I also about to propose two ports for ICD: devel/ocl-icd (at the moment version 2.2.3) and devel/opencl-icd (which is a simple port of Khronos' suggestion of their ICD). Please give me a bit time to make updated proposals ready.

I messed up a bit with the requistes and the porting of POCL 0.10 that way, as POCL definitely uses its own CL header files - still at OpenCL 1.2. The port does get into a conflict while compiling using port devel/opencl, which provides now headers for OpenCL 2.0.

I also appreciate any help.

Thanks in advance,

Oliver
Comment 13 Johannes M Dieterich freebsd_committer 2015-01-15 08:53:25 UTC
If you refer to #123 on github you opened: that one's solved according to the authors (see: https://github.com/pocl/pocl/issues/123). Indeed, I can confirm that the tests pass on my Ivy CPU using the latest shar-archive I attached and devel/llvm34 (disclaimer: as you also noted there, their testsuite by default skips a lot of tests).

Concerning the ICD and header files: I believe it would be worthwhile to contact the x11@ team what they envision now that we finally seem to be getting clover. :-)

Well, let me know if/how I can help. I have a vivid interest in being able to do my OpenCL development on FBSD instead of Linux.

Poking mva@: are you still up for getting this into the tree? That'll be awesome!
Comment 14 Koop Mast freebsd_committer 2015-02-03 13:02:04 UTC
We currently working on making ports for Clover (Mesa based OpenCL for radeon cards) and Beignet (OpenCL for intel cards) (based on llvm35 since I read that llvm34 has some regressions concerning OpenCL).

The plan is to put these into lang/ since it more of a language to talk to the GPU's then anything else. Apart from the fact that devel/ is already very crowded.

I think the software (CPU) implimentations should be moved to lang/ too, Hartman what are your thoughs on this proposed move?

I haven't looked closely into ocl-icd and opencl-icd. I can see why we might want to provide a offical khornos port but isn't it beter to just have one so there is no possible conflict between the two?
Comment 15 O. Hartmann 2015-05-02 09:41:15 UTC
Created attachment 156233 [details]
POCL 0.11-SHAR
Comment 16 O. Hartmann 2015-05-02 09:43:40 UTC
Lost the comment, sorry ...

Attached you'll find another SHAR targetting POCL 0.11. There is still an issue in test case 22, which is triggered due to the fact that FreeBSD names the Intel 64 bit architecture amd64 and in Linux, it is named x86_64. I'm on this but due to duties it will take a while.
Comment 17 O. Hartmann 2016-04-11 17:12:51 UTC
Created attachment 169196 [details]
POCL-0.13 released, utilizing OpenCL 2.0 and LLVM 3.8.0

PCOL-0.13 has been released, this is a cleanup of the previous attempts, using devel/opencl in its OCL 2.1 incarnation (not approved yet).
Comment 18 Johannes M Dieterich freebsd_committer 2016-04-14 03:39:52 UTC
FWIW, I am using pocl 0.13 to develop OpenCL code and it works well. I have used prior versions with the same success. Unlike freeocl (which is in the ports collection), this integrates nicely with ocl-icd loader and the rest of the OpenCL infrastructure we recently established.

This port would be a very good addition to the ports collection to allow wider adaption of OpenCL in FreeBSD.
Comment 19 David Kalliecharan 2016-09-03 13:48:44 UTC
Any progress on this port? I noticed that for freebsd users it is necessary to compile the OpenCL application with -pthread in CFLAGS and CXXFLAGS (not the port itself but the application a user writes). Should this be mentioned in a pkg-message?

Seen here:
http://portablecl.org/docs/html/faq.html?highlight=freebsd
Comment 20 O. Hartmann 2016-09-03 19:20:24 UTC
Created attachment 174353 [details]
pocl.diff (new port as diff as recommended by porter's handbook)

This patch is a DIFF against ports/devel and should give you the framework to install and use POCL.

I have not tested this patch thoroughly and it has been successfully working on FreeBSD 11-CURRENT and now on 12-CURRENT.

Some fixes has been introduced compared to my initial submits:
Comment 21 O. Hartmann 2016-09-03 19:34:16 UTC
(In reply to ohartman from comment #20)

(Accident with the return key ..., sorry)

1) With change from 11-RC to 12-CURRENT, I realized that in pkg-plist, the line according to 

%%DATADIR%%/kernel-amd64-portbld-freebsd%%OSREL%%.bc

has to be fixed introducing the %%OSREL%% substitution.

2) Thanks to @David Kalliecharan (comment 19) I added a a comment in pkg-message according to the developer's recommendation.

3) With >= LLVM 3.8.1  it seems that some strict C++11 checking was needed (followed the bug report on GIThub: https://github.com/pocl/pocl/issues/365.
By adding CONFIGURE_ARGS=CLANGXX_FLAGS=-std=gnu++11, a compilation error could be avoided.

I checked also CONFIGURE_ARGS=CLANGXX_FLAGS=-std=c++11, which I use now in my private port, which makes the port compile properly. Latter one makes me feel more FreeBSDish/CLANGish - sorry about the misuse of the language (English).
Comment 22 O. Hartmann 2016-09-04 07:36:17 UTC
Please also have a look at new devel/ocl-icd patch: Bug 212363

and new devel/opencl patch: Bug 212362
Comment 23 Chris Hutchinson 2016-09-09 14:31:05 UTC
Any chance this will *ever* hit the ports tree? ;-)
This port is *sorely* needed. Any of the small "nits"
that might accompany it, can be more easily sorted out
having more eyes, and helping hands available for it
once it's in the tree. No? :-)

Thanks ohartman, and everyone else who've helped
to get this in the tree!

--Chris
Comment 24 O. Hartmann 2016-09-10 12:05:14 UTC
Since I have been recently made aware by one of the "upper class" maintainer/comitter of the ports tree, that the port firt needs a proper test via poudriere and portlint to keep the quality of the tree up, I need first to test the port whether it builds correctly under "fresh and vanilla" conditions.

poudriere and jails are new to me and although setup is relatively easy, there are several pitfalls in custom setups and then someone, like me, with limited knowledge runs into problems. poudriere and jails put another level of efforts and labor to the portingwork.

I will do my best to test the port within a poudriere environment! Stay tuned.

Regards,
Oliver
Comment 25 O. Hartmann 2016-09-10 18:40:48 UTC
Testing the proposed port devel/pocl via poudriere results in an error I do not understand, since in a fresh vanilla REAL installation this error doesn't occur. The log shows :

[...]
---End make.conf---
=======================<phase: check-sanity   >============================
/!\ pocl-0.13: Makefile warnings, please consider fixing /!\

It looks like the libhwloc.so:/usr/ports/devel/hwloc depends line has an absolute port origin, make sure to remove ${PORTSDIR}/ from it.

===>  License GPLv3 accepted by the user
===>  Found saved configuration for pocl-0.13
===========================================================================
=======================<phase: pkg-depends    >============================
===>   pocl-0.13 depends on file: /usr/local/include/CL/cl.h - not found
===>   Installing existing package /packages/All/opencl-2.1.txz
/usr/ports/Mk/Scripts/do-depends.sh: /usr/local/sbin/pkg-static: not found
*** Error code 127

Stop.
make: stopped in /usr/ports/devel/pocl
[...]

The environment has been enriched with the patches for port devel/opencl ( PR 212362 ) and devel/ocl-icd ( PR 212363 ) - both ports build well without error.

I do not know what this error shown above menas - maybe someone has an idea. I guess the port devel/opencl installs the header files into a folder not found by the poudriere framework. Or its a bug in poudriere.
Comment 26 Koop Mast freebsd_committer 2016-09-13 19:55:33 UTC
Currently taking a look at this port. When I get it working I will post a new diff for you to approve.
Comment 27 Koop Mast freebsd_committer 2016-10-20 19:08:48 UTC
Created attachment 175988 [details]
Current pocl 0.13 without the sysinfo patch

I haven't finished the sysinfo patch. Mind you I'm trying to do it in such a way that it is upstreamable but maybe we should just go for making it work and worry about upstream later.
Comment 28 Johannes M Dieterich freebsd_committer 2016-11-02 18:13:33 UTC
I have added an altered version of this port to the FreeBSDDesktop bring-up ports repository at https://github.com/FreeBSDDesktop/freebsd-ports-graphics (xserver-mesa-next branch).

Please file pull requests there if you want things changed, we hope to get to a converged port that can then be added to upstream soon.
Comment 29 commit-hook freebsd_committer 2016-11-21 14:37:44 UTC
A commit references this bug:

Author: kwm
Date: Mon Nov 21 14:36:52 UTC 2016
New revision: 426722
URL: https://svnweb.freebsd.org/changeset/ports/426722

Log:
  Add pocl.

  Portable OpenCL aims to be an efficient open source (MIT-licensed)
  implementation of the OpenCL 1.2 standard. pocl uses Clang as an
  OpenCL C frontend and LLVM for the kernel compiler implementation,
  and as a portability layer. Thus, if your desired target has an LLVM
  backend, it should be able to get OpenCL support easily by using pocl.

  PR:		171914
  Submitted by:	O.Hartmann <ohartmann@walstatt.org> (based on)

Changes:
  head/lang/Makefile
  head/lang/pocl/
  head/lang/pocl/Makefile
  head/lang/pocl/distinfo
  head/lang/pocl/files/
  head/lang/pocl/files/patch-Makefile.in
  head/lang/pocl/files/patch-lib_CL_devices_cpuinfo.c
  head/lang/pocl/files/patch-scripts_Makefile.in
  head/lang/pocl/pkg-descr
  head/lang/pocl/pkg-message
  head/lang/pocl/pkg-plist
Comment 30 Koop Mast freebsd_committer 2016-11-21 14:41:29 UTC
Committed thanks!

I put it in lang/ so it together with all the other opencl "libraries".