Bug 217283 - [NEW PORT] lang/OpenCoarrays : Fortran coarrays support for gfortran
Summary: [NEW PORT] lang/OpenCoarrays : Fortran coarrays support for gfortran
Status: Closed FIXED
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Only Me
Assignee: Tijl Coosemans
URL:
Keywords:
Depends on: 217682
Blocks:
  Show dependency treegraph
 
Reported: 2017-02-22 00:11 UTC by Anton Shterenlikht
Modified: 2018-02-19 16:14 UTC (History)
0 users

See Also:


Attachments
svn diff made from /usr/ports (4.31 KB, patch)
2017-02-22 00:11 UTC, Anton Shterenlikht
no flags Details | Diff
patch (9.95 KB, patch)
2017-02-26 14:26 UTC, Tijl Coosemans
no flags Details | Diff
patch2 (10.24 KB, patch)
2017-03-04 16:03 UTC, Tijl Coosemans
no flags Details | Diff
patch3 (11.19 KB, patch)
2017-03-10 13:50 UTC, Tijl Coosemans
no flags Details | Diff
Patch tested by tijl@ (3.39 KB, patch)
2018-02-19 15:35 UTC, Anton Shterenlikht
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Anton Shterenlikht 2017-02-22 00:11:20 UTC
Created attachment 180209 [details]
svn diff made from /usr/ports

1. Add DEFAULT_VERSIONS=gcc=6 (or higher) to /etc/make.conf.
2. Rebuild net/mpich.

Then can build lang/OpenCoarrays.
Comment 1 Tijl Coosemans freebsd_committer 2017-02-26 14:26:59 UTC
Created attachment 180316 [details]
patch

I've reworked the patch.  Please give it a try, particularly the Open MPI options because I have not tested them yet.

Changes made:
- Add run dependency on bash.
- Add options for Open MPI (v1 and v2) like you asked on the mailing list.  I did not add MPICH 2 because that's an old port without upstream support.
- Replace do-build with USES=cmake:outsource and USES=fortran.  The sed commands have been replaced with USES=shebangfix.
- Remove do-install and simply use the upstream 'make install'.  This required adding files/patch-src_mpi_CMakeLists.txt so a symbolic link is created inside DESTDIR (stage directory) instead of /usr/local.  The patch also makes the link relative instead of absolute.  You may want to upstream this.
- Add files/patch-CMakeLists.txt which removes a configure test that fails with Clang.  This allows using clang as CC.  You should not upstream this because it's not their fault.  The test fails because the compiler invocation is missing -L/usr/local/lib/gcc5 so it cannot find libgcc_s in that directory and instead uses the older /usr/lib/libgcc_s.so which doesn't work.  The test uses CHECK_C_SOURCE_COMPILES which does not provide a mechanism to add -L flags (or LINK_DIRECTORIES in cmake parlance) but even if it did the -L flag should come from the FindMPI cmake module which gets it from running "mpicc -show" but it deliberately removes it.  It does pass on -Wl,-L flags so I could patch net/mpich to make "mpicc -show" output that, but this too fails with clang because when clang calls the linker, it first collects all -L flags, then it inserts -L/usr/lib and then it puts other linker flags like -Wl,-L.  Because of that -L/usr/lib the wrong libgcc_s is still used.  So right now the easiest is to remove the test.  Because of another bug report I sent a patch for lang/gcc to Gerald earlier this week which mitigates some of the problems we have with libgcc_s by making libgfortran work with /usr/lib/libgcc_s.so in many cases.  It would also help in this case, but I have yet to receive a response.
Comment 2 Anton Shterenlikht 2017-02-27 17:20:04 UTC
Tijl, many thanks for a swift and thorough check/reworking.

make install, make package fail for me:

root@20474701626e:/usr/ports/lang/OpenCoarrays # make install
===>  Installing for OpenCoarrays-1.8.4
===>   OpenCoarrays-1.8.4 depends on executable: bash - found
===>   OpenCoarrays-1.8.4 depends on executable: mpif90 - found
===>   OpenCoarrays-1.8.4 depends on executable: gfortran6 - found
===>  Checking if OpenCoarrays already installed
===>   Registering installation for OpenCoarrays-1.8.4
pkg-static: Unable to access file /usr/ports/lang/OpenCoarrays/work/stage/usr/local/include/OpenCoarrays/GNU/5.4.0/opencoarrays.mod: No such file or directory
*** Error code 74

Stop.
make[1]: stopped in /usr/ports/lang/OpenCoarrays
*** Error code 1

Stop.
make: stopped in /usr/ports/lang/OpenCoarrays

root@20474701626e:/usr/ports/lang/OpenCoarrays # make package
===>  Building package for OpenCoarrays-1.8.4
pkg-static: Unable to access file /usr/ports/lang/OpenCoarrays/work/stage/usr/local/include/OpenCoarrays/GNU/5.4.0/opencoarrays.mod: No such file or directory
*** Error code 1

Stop.
make[1]: stopped in /usr/ports/lang/OpenCoarrays
*** Error code 1

Stop.
make: stopped in /usr/ports/lang/OpenCoarrays
root@20474701626e:/usr/ports/lang/OpenCoarrays # 


I think 5.4.0 is wrong.
Instead I have:
./work/stage/usr/local/include/OpenCoarrays/GNU/6.3.1/opencoarrays.mod
./work/stage/usr/local/include/opencoarrays.mod

which are identical:

root@20474701626e:/usr/ports/lang/OpenCoarrays # diff ./work/stage/usr/local/include/OpenCoarrays/GNU/6.3.1/opencoarrays.mod ./work/stage/usr/local/include/opencoarrays.mod
root@20474701626e:/usr/ports/lang/OpenCoarrays # 

So I think

include/OpenCoarrays/GNU/5.4.0/opencoarrays.mod

is probably not needed at all.

Also, I don't understand why you include cmake files
into the installation (pkg-plist):

lib/cmake/opencoarrays/OpenCoarraysConfig.cmake
lib/cmake/opencoarrays/OpenCoarraysConfigVersion.cmake
lib/cmake/opencoarrays/OpenCoarraysTargets-%%CMAKE_BUILD_TYPE%%.cmake
lib/cmake/opencoarrays/OpenCoarraysTargets.cmake

I thought these are needed only to build the port?

Thanks

Anton
Comment 3 Anton Shterenlikht 2017-02-28 08:28:40 UTC
Do I understand correctly that this port will not
be able to build automatically by the portscluster
(or whatever it's called nowadays) because it needs
changes to /etc/make.conf?
Comment 4 Anton Shterenlikht 2017-02-28 08:53:46 UTC
When openmpi or openmpi2 are selected, then:

-- Could NOT find MPI_C (missing:  MPI_C_LIBRARIES MPI_C_INCLUDE_PATH) 
-- Could NOT find MPI_Fortran (missing:  MPI_Fortran_LIBRARIES MPI_Fortran_INCLUDE_PATH) 
CMake Error at /usr/local/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:138 (message):
  Could NOT find MPI_C (missing: MPI_C_LIBRARIES MPI_C_INCLUDE_PATH)
Call Stack (most recent call first):
  /usr/local/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:378 (_FPHSA_FAILURE_MESSAGE)
  /usr/local/share/cmake/Modules/FindMPI.cmake:616 (find_package_handle_standard_args)
  CMakeLists.txt:164 (find_package)
Comment 5 Tijl Coosemans freebsd_committer 2017-03-04 16:03:30 UTC
Created attachment 180498 [details]
patch2

- Define MPI_HOME for OPENMPI options.
- Use ${_GCC_VER} to locate opencoarrays.mod.
Comment 6 Tijl Coosemans freebsd_committer 2017-03-04 16:17:26 UTC
The two opencoarrays.mod files are the same because one is a symbolic link to the other.
The cmake files are for developers that use cmake with their projects.  They define library locations and linker flags and can be imported by other cmake projects.
There will indeed be no opencoarrays package by default.  Work has started to make GCC 5 the default so this is only temporary.
Comment 7 Anton Shterenlikht 2017-03-06 16:00:50 UTC
If mpich or openmpi are installed as well as openmpi2,
then build with openmpi2 fails:

[ 12%] Linking Fortran executable hello_multiverse
cd /usr/ports/lang/OpenCoarrays/work/.build/src/tests/integration/coarrayHelloWorld && /usr/local/bin/cmake -E cmake_link_script CMakeFiles/hello_multiverse.dir/link.txt --verbose=1
/usr/local/bin/gfortran6   -Wl,-rpath  -Wl,/usr/local/lib  -Wl,-rpath  -Wl,/usr/local/mpi/openmpi2/lib  -Wl,--enable-new-dtags  -Wl,-rpath=/usr/local/lib/gcc6  -L/usr/local/lib/gcc6 -B/usr/local/bin -fstack-protector -O -O2 -pipe -march=bdver2 -mtune=bdver2 -funroll-loops --param max-unroll-times=4 -ftree-vectorize -Wl,-rpath=/usr/local/lib/gcc6 -O3 -DNDEBUG -O3 CMakeFiles/hello_multiverse.dir/hello_multiverse.F90.o  -o hello_multiverse -Wl,-rpath,/usr/local/lib:/usr/local/mpi/openmpi2/lib  ../../../mpi/libcaf_mpi.a /usr/local/lib/libmpi.so /usr/local/mpi/openmpi2/lib/libmpi_usempi.so /usr/local/mpi/openmpi2/lib/libmpi_mpifh.so /usr/local/lib/libmpi.so /usr/local/mpi/openmpi2/lib/libmpi_usempi.so /usr/local/mpi/openmpi2/lib/libmpi_mpifh.so 
/usr/local/bin/ld: warning: libmpi.so.20, needed by /usr/local/mpi/openmpi2/lib/libmpi_usempi.so, may conflict with libmpi.so.12
/usr/local/bin/ld: warning: libmpi.so.20, needed by /usr/local/mpi/openmpi2/lib/libmpi_usempi.so, may conflict with libmpi.so.12
/usr/local/bin/ld: ../../../mpi/libcaf_mpi.a(mpi_caf.c.o): undefined reference to symbol 'ompi_mpi_character'
/usr/local/mpi/openmpi2/lib/libmpi.so.20: error adding symbols: DSO missing from command line
collect2: error: ld returned 1 exit status
*** [src/tests/integration/coarrayHelloWorld/hello_multiverse] Error code 1

make[4]: stopped in /usr/ports/lang/OpenCoarrays/work/.build


libmpi.so.12 are installed by mpich and openmpi:

# find /usr/local/ -name "libmpi.so.12"
/usr/local/lib/libmpi.so.12
/usr/local/mpi/openmpi/lib/libmpi.so.12

Should this be addressed in the port?
Some check for mpich or openmpi, and if these
are installed, warn the user that building with
openmpi2 will fail, or something like this?

Otherwise build/install etc. seem fine now.

There are multiple test failures.
I'm working with opencoarrays folk about that,
but could take a while. I suspect linuxisms.

I'm happy with the port now.

Thanks for your help

Anton
Comment 8 Anton Shterenlikht 2017-03-06 16:22:43 UTC
Just noticed this warning:

Installing OpenCoarrays-1.8.4...
===> SECURITY REPORT: 
      This port has installed the following world-writable files/directories.
/usr/local/bin/caf
/usr/local/bin/cafrun

      If there are vulnerabilities in these programs there may be a security
      risk to the system. FreeBSD makes no guarantee about the security of
      ports included in the Ports Collection. Please type 'make deinstall'
      to deinstall the port if this is a concern.

      For more information, and contact details about the security
      status of this software, see the following webpage: 
http://www.opencoarrays.org/

# ls -al /usr/local/bin/caf /usr/local/bin/cafrun 
-rwxrwxrwx  1 root  wheel  7027 Mar  6 15:52 /usr/local/bin/caf
-rwxrwxrwx  1 root  wheel  3564 Mar  6 15:52 /usr/local/bin/cafrun

Should we address this?

Looks like most files under /usr/local/bin are 444 or 644.
Comment 9 Tijl Coosemans freebsd_committer 2017-03-10 13:50:55 UTC
Created attachment 180697 [details]
patch3

Remove write permissions from caf and cafrun.
Comment 10 Anton Shterenlikht 2017-03-10 17:29:46 UTC
cool, seems fine now.

Thanks

Anton
Comment 11 commit-hook freebsd_committer 2017-03-12 20:26:04 UTC
A commit references this bug:

Author: tijl
Date: Sun Mar 12 20:25:30 UTC 2017
New revision: 436025
URL: https://svnweb.freebsd.org/changeset/ports/436025

Log:
  Add lang/OpenCoarrays, an implementation of Fortran coarrays using MPI.

  OpenCoarrays is an open-source software project that supports the coarray
  Fortran (CAF) parallel programming features of the Fortran 2008 standard
  and several features proposed for Fortran 2015.

  WWW: http://www.opencoarrays.org/

  PR:		217283
  Submitted by:	Anton Shterenlikht <mexas@bris.ac.uk>

Changes:
  head/lang/Makefile
  head/lang/OpenCoarrays/
  head/lang/OpenCoarrays/Makefile
  head/lang/OpenCoarrays/distinfo
  head/lang/OpenCoarrays/files/
  head/lang/OpenCoarrays/files/patch-CMakeLists.txt
  head/lang/OpenCoarrays/files/patch-src_mpi_CMakeLists.txt
  head/lang/OpenCoarrays/files/patch-src_mpi_mpi__caf.c
  head/lang/OpenCoarrays/pkg-descr
  head/lang/OpenCoarrays/pkg-plist
Comment 12 Anton Shterenlikht 2018-02-19 15:35:29 UTC
Created attachment 190805 [details]
Patch tested by tijl@

passes all tests on FreeBSD 12. Some test failures on FreeBSD 11.
Comment 13 Tijl Coosemans freebsd_committer 2018-02-19 16:14:24 UTC
(In reply to Anton Shterenlikht from comment #12)
Anton, I suspect you meant to attach this to bug 222585.