Bug 244296 - math/openblas: Need to resolve conflicts with cblas and lapacke
Summary: math/openblas: Need to resolve conflicts with cblas and lapacke
Status: Closed FIXED
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Some People
Assignee: freebsd-ports-bugs (Nobody)
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2020-02-22 01:38 UTC by Jason W. Bacon
Modified: 2021-06-13 17:50 UTC (History)
8 users (show)

See Also:
bugzilla: maintainer-feedback? (phd_kimberlite)


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Jason W. Bacon freebsd_committer freebsd_triage 2020-02-22 01:38:31 UTC
I see that openblas now cannot be installed alongside cblas or lapacke.

This is a big problem for many sites as it means some users cannot install all the dependent software they need side-by-side.  This is especially problematic for multi-user installations such as HPC clusters and lab workstations.

I wonder if we could patch out the bundled cblas and lapacke headers and add cblas and lapacke as dependent ports instead.  They should be compatible, if not identical.

Thanks,

    JB
Comment 1 Thierry Thomas freebsd_committer freebsd_triage 2020-02-22 09:26:19 UTC
Another possibility could be to set the default to openblas for as many ports as possible, so that packages won't conflict.

At the moment Mk/Uses/blaslapack.mk default to netlib, but we can change that. We could also modify it to handle cblas, with an extra argument.
Comment 2 Jason W. Bacon freebsd_committer freebsd_triage 2020-02-22 14:34:02 UTC
(In reply to Thierry Thomas from comment #1)

I think that would be a good idea, though there will always be a need to allow different BLAS/LAPACK implementations to coexist.  For instance, plink2 (https://github.com/outpaddling/freebsd-ports-wip/tree/master/plink2) currently crashes when linked with openblas but works fine with netlib.  Since BLAS is not a bottleneck for this program, there's no reason not to make it available using netlib (which is slow) while we try to diagnose the issue with openblas.

While the various implementations are meant to be fully compatible, the reality is that there will always be bugs and minor differences.  Allowing users to work around them by using another implementation for a while will keep them happy and productive with FreeBSD.

I was planning to save this for a separate conversation, but since we're on the subject of systemic solutions, I'm preparing to commit the work of T. Orgis on an interchangeable BLAS interface for pkgsrc:

https://wip.pkgsrc.org/cgi-bin/gitweb.cgi?p=pkgsrc-wip.git;a=blob;f=mk/blas.buildlink3.mk;h=e09d41644b7d6e3eb183b03b8a40604ef632efe4;hb=HEAD

This system allows the sysadmin to specify one or more preferred BLAS implementations in mk.conf (equiv to make.conf on FreeBSD) and also allows the package maintainer to specify which implementations are acceptable in the package Makefile.

After some extensive discussion, we ended up with a pretty elegant and flexible solution.
Comment 3 Koichiro Iwao freebsd_committer freebsd_triage 2020-03-31 04:40:51 UTC
This also affect me.
Comment 4 Koichiro Iwao freebsd_committer freebsd_triage 2020-03-31 04:43:43 UTC
math/py-numpy depends on both openblas and cblas. It cannot be installed unless this issue is solved. 
> ===>>> math/py-numpy >> (3)
> 
> ===>>> The following actions will be taken if you choose to proceed:
> 	Install math/py-numpy
>	Install math/cblas
> 	Install math/suitesparse
> 	Install math/openblas
> 
> ===>>> Proceed? y/n [y]
Comment 5 Harry NEWTON 2020-06-02 07:13:49 UTC
Affects me too.  Cannot install numpy or scipy.

Is it related to the changes made for https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=243497 ?
Comment 6 Dave Hayes 2020-06-13 22:53:43 UTC
And me. Note:

pkg-static: openblas-0.3.7_2,1 conflicts with cblas-1.0_12 (installs files into the same place).  Problematic file: /usr/local/include/cblas.h
Comment 7 John Hein 2020-06-14 17:44:02 UTC
(In reply to Koichiro Iwao from comment #4)
So that looks like portmaster output.  Somehow the options for one of your dependent ports or py-numpy itself is configure such that both openblas and cblas are needed.

In the default case for py-numpy (tested by a 'poudriere testport' run for math/py-numpy), openblas is used, but cblas is not.  So something in your settings and/or installed ports is pulling in cblas while the py-numpy port depends on openblas by default.

'make -C /usr/ports/math/py-numpy showconfig' will show the options configured for py-numpy (look to see which *blas option is configured).  But even if your py-numpy installation specifies OPENBLAS, say, other dependent ports may have NETLIB=on (the NETLIB option for py-numpy currently pulls in cblas).

'grep -r NETLIB /var/db/ports' will search the options for the ports you have built yourself.

This will show NETLIB & OPENBLAS settings for all installed packages:

pkg query -a '%n %Ok=%Ov' | egrep 'NETLIB|OPENBLAS'

'pkg info -dr cblas' will show the dependencies and requirements for cblas on your system.

Until someone makes changes such that math/cblas and math/openblas and math/lapacke can co-exist (maybe separating headers to a separate port, for instance), one can pick and use just one.  If that's not possible (some features of more than one of the conflicting ports are needed), then someone will need to find a way for them to co-exist.

If you don't know that you need more than one of the conflicting ports, try just using one, if possible.
Comment 8 Thierry Thomas freebsd_committer freebsd_triage 2020-06-24 15:50:52 UTC
I'm working on an upgrade of Netlib ports, and it should take care of this problem.
Comment 9 Thierry Thomas freebsd_committer freebsd_triage 2020-08-04 17:19:19 UTC
Back to pool.
Comment 10 Thierry Thomas freebsd_committer freebsd_triage 2021-06-13 17:50:17 UTC
Fixed when upgrading cblas and lapacke to 3.9.1: see PR 247542.