Bug 251894

Summary: science/hdf5: repocopy to science/hdf5-110 and then upgrade to 1.12
Product: Ports & Packages Reporter: Thierry Thomas <thierry>
Component: Individual Port(s)Assignee: Po-Chuan Hsieh <sunpoet>
Status: Open ---    
Severity: Affects Many People CC: bofh, jwb, makc, portmgr, rhurlin
Priority: --- Flags: bugzilla: maintainer-feedback? (sunpoet)
Version: Latest   
Hardware: Any   
OS: Any   
See Also: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=234542
Bug Depends on: 252968, 252965, 252966, 252967, 252969, 252970    
Bug Blocks: 252565, 252566    
Description Flags
To be applied after the repocopy from science/hdf5 to science/hdf5-112
Patch to upgrade science/hdf5 to 1.12
Patch to keep hdf5 1.10 in // with 1.12
Patch to upgrade science/hdf5 to 1.12
Unique patch for science/hdf5-110, science/hdf5 and some consumers
thierry: maintainer-approval+
Unique patch for hdf5, hdf5-110 ans some consumers
thierry: maintainer-approval+
Patch to upgrade science/hdf5 to 1.12 thierry: maintainer-approval?

Description Thierry Thomas freebsd_committer 2020-12-16 15:49:23 UTC
HDF5 has been upgraded to 1.12.0: release notes available at

Unfortunately, these new versions break the compatibility with the previous ones, as this was the case from 1.8 to 1.10. It proposes an option to keep some compatibility with 1.10, DEFAULT_API_VERSION="v110", and this flag is defined in the attached patch.

Nevertheless, ATM some consumers cannot be built against this version, e.g. french/med, this is why I don't propose a direct upgrade to science/hdf5, but to repocopy it as science/hdf5-112, like we did for hdf5-18.

A migration guide is available at:

Note 1: two tarballs are available, and they produce two different packages. I have chosen the cmake version, which seems more complete.

Note 2: MAKE_JOBS_UNSAFE seems not more useful on my machines.

Note 3: the option FORTRAN is now selected by default, as requested in PR 225070, so that the FORTRAN interface will be available in packages.

Proposed migration path for FreeBSD:
- add this port as hdf5-112;
- migrate the different consumers to this new version on a per-port basis, if possible;
- when the major consumers will be migrated, switch hdf5 to hdf5-110 for the last ones, and switch hdf5-112 to hdf5.
Comment 1 Thierry Thomas freebsd_committer 2020-12-16 15:50:21 UTC
Created attachment 220626 [details]
To be applied after the repocopy from science/hdf5 to science/hdf5-112
Comment 2 Max Brazhnikov freebsd_committer 2020-12-23 15:55:15 UTC
(In reply to Thierry Thomas from comment #0)
This double switch and hdf5 port naming looks too confusing for me. Could you please update hdf5 directly to the new version and add hdf5-$legacy_ver for ports, that don't support latest hdf5?
Comment 3 Thierry Thomas freebsd_committer 2020-12-23 19:07:53 UTC
(In reply to Max Brazhnikov from comment #2)

This is not a bad idea, but in this case an exp-run would be necessary.

sunpoet@, any thoughts?
Comment 4 Po-Chuan Hsieh freebsd_committer 2021-01-06 20:59:56 UTC
(In reply to Thierry Thomas from comment #3)

I agree with Max. And we need an exp-run to see what ports will stay with hdf5 1.10.
Comment 5 Thierry Thomas freebsd_committer 2021-01-09 10:08:07 UTC
Created attachment 221416 [details]
Patch to upgrade science/hdf5 to 1.12
Comment 6 Thierry Thomas freebsd_committer 2021-01-09 10:10:34 UTC
Created attachment 221417 [details]
Patch to keep hdf5 1.10 in // with 1.12

This patch (to be applied after a repocopy of science/hdf5 to science/hdf5-110) allows to install hdf5 1.12 and hdf5 1.10 without conflict.
Comment 7 Thierry Thomas freebsd_committer 2021-01-09 10:12:08 UTC
(In reply to Sunpoet Po-Chuan Hsieh from comment #4)

Here we are! The two attached patches provide hdf5-1.12.0,1 and hdf5-110-1.10.6,1.
Comment 8 Thierry Thomas freebsd_committer 2021-01-09 15:40:48 UTC
Created attachment 221422 [details]
Patch to upgrade science/hdf5 to 1.12

This patch is better than the previous one: it installs the headers directly under $PREFIX/include to ease the migration.
Comment 9 Thierry Thomas freebsd_committer 2021-01-10 22:00:49 UTC
Migration notes:

- in 1.12 h5cc, h5c++ and h5fc are just wrappers around pkg-config, and they take no more argument

	. `h5cc -show' can be replaced by `pkg-config --cflags --libs hdf5'
	  (same for h5fc and h5cc)

	. `h5cc -showconfig' can be replaced by `cat $LOCALBASE/lib/libhdf5.settings'

- H5_USE_110_API is set to allow the use of the previous API, but H5_NO_DEPRECATED_SYMBOLS must not be set!
Comment 10 Antoine Brodin freebsd_committer 2021-01-11 08:46:05 UTC
Can you provide a patch that covers everything for the exp-run?  (instead of multiple patches in multiple PR)
Comment 11 Thierry Thomas freebsd_committer 2021-01-11 19:14:37 UTC
Created attachment 221475 [details]
Unique patch for science/hdf5-110, science/hdf5 and some consumers

Attached is an unique patch.

Note: I only tested some ports that I maintain and some related ones, but more failures are expected to be found by the exp-run!
Comment 12 Antoine Brodin freebsd_committer 2021-01-20 19:26:31 UTC
There is a problem,  hdf5 seems unfetchable:

Comment 13 Thierry Thomas freebsd_committer 2021-01-20 21:25:42 UTC
(In reply to Antoine Brodin from comment #12)

They changed their directories! 1.12.0 is still available from

MASTER_SITES must be modified to:

MASTER_SITES=	https://support.hdfgroup.org/ftp/HDF5/releases/hdf5-${PORTVERSION:R}/hdf5-${PORTVERSION}/src/

but this the same tarball (same date, size and SHA256)
Comment 15 Antoine Brodin freebsd_committer 2021-01-22 13:26:22 UTC
(In reply to Antoine Brodin from comment #14)
wkhtmltopdf may be a false positive
Comment 16 Thierry Thomas freebsd_committer 2021-01-24 09:56:05 UTC
Created attachment 221869 [details]
Unique patch for hdf5, hdf5-110 ans some consumers

hdfgroup.org reorganized their directories: this new patch updates MASTER_SITES accordingly.
Comment 17 Jason W. Bacon freebsd_committer 2021-01-24 12:53:14 UTC

<<<ROOT@coral.acadix>>> /usr/ports/wip/hdf5 1017 # make distclean build
===>  Cleaning for hdf5-1.12.0,1
===>  Deleting distfiles for hdf5-1.12.0,1
===>  License BSD4CLAUSE accepted by the user
===>   hdf5-1.12.0,1 depends on file: /usr/local/sbin/pkg - found
=> CMake-hdf5-1.12.0.tar.gz doesn't seem to exist in /usr/ports/distfiles/.
=> Attempting to fetch https://support.hdfgroup.org/ftp/HDF5/releases/hdf5-1.12/CMake-hdf5-1.12.0/src/CMake-hdf5-1.12.0.tar.gz
fetch: https://support.hdfgroup.org/ftp/HDF5/releases/hdf5-1.12/CMake-hdf5-1.12.0/src/CMake-hdf5-1.12.0.tar.gz: Not Found
=> Attempting to fetch http://distcache.FreeBSD.org/ports-distfiles/CMake-hdf5-1.12.0.tar.gz
fetch: http://distcache.FreeBSD.org/ports-distfiles/CMake-hdf5-1.12.0.tar.gz: Not Found
=> Couldn't fetch it - please try to retrieve this
=> port manually into /usr/ports/distfiles/ and try again.
*** Error code 1

make[1]: stopped in /usr/ports/wip/hdf5
*** Error code 1

make: stopped in /usr/ports/wip/hdf5

<<<ROOT@coral.acadix>>> /usr/ports/wip/hdf5 1018 # cat distinfo
TIMESTAMP = 1608044656
SHA256 (CMake-hdf5-1.12.0.tar.gz) = 01b9c01c45cc8c66da86e69c510e17f3cff0706a65d8683cd86af405eaf75397
SIZE (CMake-hdf5-1.12.0.tar.gz) = 13950636
Comment 18 Thierry Thomas freebsd_committer 2021-01-24 13:09:58 UTC
(In reply to Jason W. Bacon from comment #17)
Oh yes, hdfgroup.org reorganized their directories, and I only updated the unique patch!
Here is the update for the single patch.
Comment 19 Thierry Thomas freebsd_committer 2021-01-24 13:11:26 UTC
Created attachment 221874 [details]
Patch to upgrade science/hdf5 to 1.12

Modify MASTER_SITES after upstream's reorganization.
Comment 20 Po-Chuan Hsieh freebsd_committer 2021-01-26 19:52:38 UTC
(In reply to Thierry Thomas from comment #19)

Thierry, thanks for your work. It's no doubt that you spent lots of time on it.

I tried to understand your patch and I got some questions:
1. Why do you use 1.10 API by default for hdf5 1.12?
2. What is the purpose of extra .pc files, e.g. hdf5-1.12.0.pc (symlink to hdf5.pc)?

Based on your work, I prepared a WIP patch [1] which is easier to read (at least for me). I still need some time to refine it.

The main difference is as follows:
1. For science/hdf5, use 1.12 API by default rather than 1.10
2. Enable zlib, otherwise netcdf would fail during configure
3. Disable FORTRAN by default

[1] https://people.FreeBSD.org/~sunpoet/patch/hdf5.txt
Comment 21 Thierry Thomas freebsd_committer 2021-01-26 21:15:06 UTC
(In reply to Sunpoet Po-Chuan Hsieh from comment #20)

1) I used 1.10 API by default for hdf5 1.12 to ease the migration, so that most of the ports can depend directly on the latest port. Of course you can change it but I'm afraid that many more ports will have to depend on hdf5-110.

2) The port installs hdf5-1.12.0.pc, and I guess that it's not very useful, it's why I created the symlink (see PR 234542 for some use case).

3) FORTRAN by default has been requested in PR 225070. It's an overhead, but not very important for packages, and can be disabled by ports users.

Of course, you are the maintainer and know this port better than me: choose what seems the best solution!
Comment 22 commit-hook freebsd_committer 2021-01-27 11:14:30 UTC
A commit references this bug:

Author: makc
Date: Wed Jan 27 11:14:18 UTC 2021
New revision: 563043
URL: https://svnweb.freebsd.org/changeset/ports/563043

  - Add patch from AUR to fix build with upcoming hdf5-1.12

  PR:		251894
  PR:		252967

Comment 23 commit-hook freebsd_committer 2021-02-23 15:46:53 UTC
A commit references this bug:

Author: makc
Date: Tue Feb 23 15:46:38 UTC 2021
New revision: 566405
URL: https://svnweb.freebsd.org/changeset/ports/566405

  - Add support for HDF5 v1.12
  - while here replace sed with patch

  PR:		251894

Comment 24 Max Brazhnikov freebsd_committer 2021-07-27 18:36:21 UTC
(In reply to Po-Chuan Hsieh from comment #20)
> [1] https://people.FreeBSD.org/~sunpoet/patch/hdf5.txt

This patch is gone. Could you prepare patch for exp-run? I'd like to know ports which need fix for 1.12 API.