Bug 238008 - math/fftw3: WIth OPENMP=ON, OPENMPI-ON it installs lib/libfftw3_mpi.so with unresolved ompi_xx and other symbols
Summary: math/fftw3: WIth OPENMP=ON, OPENMPI-ON it installs lib/libfftw3_mpi.so with u...
Status: Closed Not A Bug
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Only Me
Assignee: Jason E. Hale
URL: https://github.com/FFTW/fftw3/issues/169
Keywords:
Depends on:
Blocks:
 
Reported: 2019-05-20 20:17 UTC by Yuri Victorovich
Modified: 2019-05-21 19:05 UTC (History)
0 users

See Also:
bugzilla: maintainer-feedback? (jhale)


Attachments
Testcase: pfft.shar (2.97 KB, text/plain)
2019-05-20 20:17 UTC, Yuri Victorovich
no flags Details
Add rpath for openmpi (391 bytes, patch)
2019-05-21 04:09 UTC, Jason E. Hale
no flags Details | Diff
fftw3-add-openmpi-lib.patch (413 bytes, patch)
2019-05-21 05:29 UTC, Yuri Victorovich
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Yuri Victorovich freebsd_committer freebsd_triage 2019-05-20 20:17:42 UTC
Created attachment 204491 [details]
Testcase: pfft.shar

Failure:
> configure:20330: cc -o conftest -O2 -pipe -fno-omit-frame-pointer -fstack-protector-strong -isystem /usr/local/include -fno-strict-aliasing -fno-omit-frame-pointer -isystem /usr/local/include -fstack-protector-strong -L/usr/local/lib conftest.c -lfftw3_mpi -lfftw3 -lm -lmpi >&5
> ld: error: /usr/local/lib/libfftw3_mpi.so: undefined reference to ompi_mpi_op_sum
> ld: error: /usr/local/lib/libfftw3_mpi.so: undefined reference to ompi_mpi_comm_null
> ld: error: /usr/local/lib/libfftw3_mpi.so: undefined reference to ompi_mpi_unsigned
> ld: error: /usr/local/lib/libfftw3_mpi.so: undefined reference to MPI_Comm_f2c
> ld: error: /usr/local/lib/libfftw3_mpi.so: undefined reference to ompi_mpi_op_land
> ld: error: /usr/local/lib/libfftw3_mpi.so: undefined reference to ompi_mpi_unsigned_long
> ld: error: /usr/local/lib/libfftw3_mpi.so: undefined reference to ompi_mpi_op_lor


See the attached new port math/pfft as a testcase.
Comment 1 Jason E. Hale freebsd_committer freebsd_triage 2019-05-20 23:14:21 UTC
This seems to be a problem of the configure script not finding the OpenMPI libraries. I don't think this really has anything to do with FFTW3 itself.

I added the following to your Makefile and it configured and built for me:
CFLAGS+= -I${LOCALBASE}/mpi/openmpi/include                  
LDFLAGS+= -L${LOCALBASE}/mpi/openmpi/lib
Comment 2 Yuri Victorovich freebsd_committer freebsd_triage 2019-05-21 02:39:44 UTC
Sure these symbols can be solved at the user's level, but libraries, once installed, normally should have all their symbols resolved through the linked libraries.
Comment 3 Yuri Victorovich freebsd_committer freebsd_triage 2019-05-21 02:50:27 UTC
Added the upstream bug report URL.
Comment 4 Jason E. Hale freebsd_committer freebsd_triage 2019-05-21 02:52:19 UTC
(In reply to Yuri Victorovich from comment #2)
The linker still needs to know where to find -lmpi. It's on the first line of the error from comment #1. I couldn't even get past the check for MPI_Init without that hack. I'm not sure how you made it further in the configure process.

checking for function MPI_Init... no
checking for function MPI_Init in -lmpi... no
checking for function MPI_Init in -lmpich... no
configure: error: in `/usr/home/jhale/src/freebsd/ports/math/pfft/work/pfft-1.0.8-alpha-66-ge4cfcf9':
configure: error: PFFT requires an MPI C compiler.
Comment 5 Yuri Victorovich freebsd_committer freebsd_triage 2019-05-21 03:25:52 UTC
On my system it fails differently:

checking for fftw_execute in -lfftw3... yes
checking for fftw_mpi_init in -lfftw3_mpi... no
configure: error: You do not seem to have the MPI part of the  FFTW-3.3 library installed.
      You can download it from http://www.fftw.org. If you have installed FFTW-3.3,
      make sure that this configure script can find it. See ./configure --help
      for more information.

Then config.log has the errors that libfftw3_mpi.so has unresolved symbols.

Possibly your system doesn't have MPI installed?
Comment 6 Jason E. Hale freebsd_committer freebsd_triage 2019-05-21 03:47:38 UTC
(In reply to Yuri Victorovich from comment #5)
Of course I have MPI installed. Otherwise, I wouldn't have been able to get it to eventually build.

Here is the output from config.log with your original port:
configure:4847: result: no
configure:4818: checking for function MPI_Init in -lmpi
configure:4840: cc -o conftest -O2 -pipe -march=broadwell  -fstack-protector-strong -isystem /usr/local/include -fno-strict-aliasing  -isystem /usr/local/include  -fstack-protector-strong -L/usr/local/lib  conftest.c -lmpi  >&5
ld: error: unable to find library -lmpi
cc: error: linker command failed with exit code 1 (use -v to see invocation)

You can see that the linker can't find -lmpi. This information needs to be supplied to the linker:
CFLAGS+= -I${LOCALBASE}/mpi/openmpi/include
LDFLAGS+= -L${LOCALBASE}/mpi/openmpi/lib

This was your error from comment #1:
configure:20330: cc -o conftest -O2 -pipe -fno-omit-frame-pointer -fstack-protector-strong -isystem /usr/local/include -fno-strict-aliasing -fno-omit-frame-pointer -isystem /usr/local/include -fstack-protector-strong -L/usr/local/lib conftest.c -lfftw3_mpi -lfftw3 -lm **-lmpi** >&5

Again, it is trying to link to -lmpi without telling the linker where -lmpi is. Nowhere on that line is there a path to the mpi library.

You port is missing dependencies and needs USES=libtool, BTW.
Comment 7 Jason E. Hale freebsd_committer freebsd_triage 2019-05-21 04:09:18 UTC
Created attachment 204499 [details]
Add rpath for openmpi

Can you try this patch with fftw3 and see if it helps your particular situation. I still can't get past the initial check for MPI without the LDFLAGS/CFLAGS, though.
Comment 8 Yuri Victorovich freebsd_committer freebsd_triage 2019-05-21 05:29:15 UTC
Created attachment 204500 [details]
fftw3-add-openmpi-lib.patch

(In reply to Jason E. Hale from comment #7)

This patch didn't help. But I am attaching another patch that helped.

The unwanted small side-effect is that it makes all shared libraries to be linked with libmpi.so, not just libfftw3_mpi.so

This is why this patch is only a workaround, and this should really be fixed by the upstream.

Additionally please note that fftw3 installs .pc files into a wrong location: mpi/openmpi/lib/pkgconfig, instead of a standard libdata/pkgconfig location.

Otherwise, fftw3 should use $(pkg-config --libs ompi-c) for LDFLAGS for libfftw3_mpi.so
Comment 9 Jason E. Hale freebsd_committer freebsd_triage 2019-05-21 12:06:04 UTC
(In reply to Yuri Victorovich from comment #8)

Here's the underlying problem: MPI code is supposed to be compiled and linked with the MPI compiler wrappers (mpicc). That is what is supposed to be adding -lmpi, etc. What fftw3 does currently is correct. You can read their documentation on it here:
http://www.fftw.org/fftw3_doc/Linking-and-Initializing-MPI-FFTW.html

I added to your Makefile (no LDFLAGS/CFLAGS):
MPIDIR?= ${LOCALBASE}/mpi/openmpi
CC= ${MPIDIR}/bin/mpicc

It configured and compiled with no modifications to fftw3. I think this is the approach you should be using.

As far as the pkgconfig file goes, I am not seeing the installation go awry on my end. No sure how it ended up in the wrong directory on your system.
Comment 10 Yuri Victorovich freebsd_committer freebsd_triage 2019-05-21 16:44:13 UTC
(In reply to Jason E. Hale from comment #9)

mpicc comes from mpich, another implementation of MPI:
> $ pkg which /usr/local/bin/mpicc
> /usr/local/bin/mpicc was installed by package mpich-3.2.1_4

Here we talk about the OpenMPI implementation. mpich and OpenMPI can't be mixed.

Additionally, all mpicc does (assuming the use of mpich, not OpenMPI) is using the same compiler and compilation options that were used when it was built itself. It isn't required, in case of mpich.
Comment 11 Yuri Victorovich freebsd_committer freebsd_triage 2019-05-21 19:05:37 UTC
Jason,

Thanks for your help.

I'll deal with this at the upstream level where is the right place to solve this anyway.


Best,
Yuri