Every time portupgrade replaces the ICU port, the binaries trying to use libicuuc.so.* and other bits installed by that port break. This is probably because the ICU port still uses not only major, but also the minor library versions -- something, portupgrade is not handling well. For example, although I have the libicuio.so.59.1 in ${LOCALBASE}/lib/compat/pkg/ now, it is not being found by anything, because other binaries look for it with just .59 (without the .1). For example: Shared object "libicuuc.so.59" not found, required by "libQt5WebKit.so.5" Once I create the necessary symlinks, things begin to work again. Also, and probably for the same reason, portupgrade misses these other packages -- and fails to rebuild them when upgrading ICU -- even when run with `-r -R`. This problem may be happening with other ports as well, but ICU is used by so many, lots of people are affected...
Using portupgrade myself, I do not have issues with icu lately. So I think it's not a portupgrade issue. In general, all ports that depend on icu need to be rebuild after an update of icu. What I usually do though after running portupgrade is a "rm -rf /usr/local/lib/compat/pkg/*" for removing old libraries, a "pkg autoremove" for removing build dependencies and a "portsclean -C -D -DD" to clean the ports.
> all ports that depend on icu need to be rebuild after an update of icu. Yes, they need to be rebuilt. But they aren't -- unless one _manually_ searches for all such packages and explicitly lists them for forced rebuild. And I suspect, that's because portupgrade is confused by ICU's libraries using both MAJOR and MINOR numbers.
Last update of icu (on 9-11-2017) went fine here using portupgrade, all dependent ports were rebuilt automagically. But iirc you do need to clean old libraries in /usr/local/lib/compat/pkg/ to prevent ports being built against the old ones.
> you do need to clean old libraries in /usr/local/lib/compat/pkg/ Well, I've never done this -- nor heard of it being mandatory -- but all other ports upgrade fine, whereas ICU is always a minefield. Maybe, the actual portupgrade AUTHORS ought to chime in, instead of us, mere USERS arguing...
(In reply to Marco Beishuizen from comment #3) > Last update of icu (on 9-11-2017) went fine here using portupgrade, all > dependent ports were rebuilt automagically. But iirc you do need to clean > old libraries in /usr/local/lib/compat/pkg/ to prevent ports being built > against the old ones. This shouldn't be needed as that path comes last in the search list. Portupgrade removes files from there when appropriate / with portsclean -L.
Try this patch for a week or two and let me know the results please. Also weave in some portsclean -L into your workflow to clean out old libraries. The patch may cause other unexpected problems but I think it's fine. Like orphaned symlinks with portsclean -L, or something linking against /usr/local/lib/compat/pkg. https://people.freebsd.org/~bdrewery/patches/portupgrade-pr-223943.diff
(In reply to Bryan Drewery from comment #6) > Try this patch for a week or two Thanks, I shall. But this patch seems to be limited to the symlink-creation, is it not? What about detecting the dependencies -- so that www/qt5-webkit is rebuilt, whenever ICU is rebuilt, for example?
(In reply to Mikhail T. from comment #7) > (In reply to Bryan Drewery from comment #6) > > Try this patch for a week or two > > Thanks, I shall. But this patch seems to be limited to the symlink-creation, > is it not? What about detecting the dependencies -- so that www/qt5-webkit > is rebuilt, whenever ICU is rebuilt, for example? That's already covered in Bug 217284.
Has this patch been working ok?
(In reply to Bryan Drewery from comment #9) Yes, I think it is working. Thank you.
A commit references this bug: Author: bdrewery Date: Fri Mar 9 19:32:18 UTC 2018 New revision: 463994 URL: https://svnweb.freebsd.org/changeset/ports/463994 Log: Update to 2.4.16. - Preserve shlib symlink [1] PR: 223943 [1] Changes: head/ports-mgmt/portupgrade/Makefile head/ports-mgmt/portupgrade/distinfo head/ports-mgmt/portupgrade-devel/Makefile head/ports-mgmt/portupgrade-devel/distinfo
(In reply to commit-hook from comment #11) Unfortunately, this breaks forced reinstall (with -f option) of (at least) x11/nvidia-driver. Reverting this fixed the issue. Maybe more ports would be affected. Error messages with ports-mgmt/portupgrade-devel are as below. (ports-mgmt/portupgrade is not tested.) Output of `portupgrade -f -b x11/nvidia-driver` (last portion): ---> Build of x11/nvidia-driver ended at: Sat, 10 Mar 2018 22:55:31 +0900 (consumed 00:00:16) ---> Uninstallation of nvidia-driver-390.25 started at: Sat, 10 Mar 2018 22:55:31 +0900 ---> Fixing up dependencies before creating a package ---> Backing up the old version ---> Uninstalling the old version [Reading data from pkg(8) ... - 1335 packages found - done] ---> Deinstalling 'nvidia-driver-390.25' ---> Preserving /compat/linux/usr/lib/libEGL_nvidia.so.0 as /usr/local/lib/compat/pkg/libEGL_nvidia.so.0 cp: symlink: libEGL_nvidia.so.390.25: File exists Copy failed! [Reading data from pkg(8) ... - 1335 packages found - done] ** Listing the failed packages (-:ignored / *:skipped / !:failed) ! nvidia-driver-390.25 (copy failed) ---> Uninstallation of nvidia-driver-390.25 ended at: Sat, 10 Mar 2018 22:57:04 +0900 (consumed 00:01:33) ---> Reinstallation of x11/nvidia-driver ended at: Sat, 10 Mar 2018 22:57:04 +0900 (consumed 00:01:49) ---> ** Upgrade tasks 1: 0 done, 0 ignored, 0 skipped and 1 failed ---> Listing the results (+:done / -:ignored / *:skipped / !:failed) ! x11/nvidia-driver (nvidia-driver-390.25) (uninstall error) ---> Packages processed: 0 done, 0 ignored, 0 skipped and 1 failed ---> Session ended at: Sat, 10 Mar 2018 22:57:04 +0900 (consumed 00:02:00)
Reason of the error is this change in /usr/local/sbin/pkg_glob: +- if system('/bin/cp', '-pf', file, dest) ++ if system('/bin/cp', '-Rpf', file, dest) I don't understand what this has to do, -R with symbolic link doesn't make sense. Try to remove it and it should work better, at least it'll overwrite existing link without error. Preserving symbolic links however will never work as portupgrade calls portsclean -QL at the end: ---> Cleaning out obsolete shared libraries man portsclean -L --libclean Clean out old, duplicate and/or orphaned shared libraries. portsclean first deletes duplicate shared libraries where appropriate, then puts away old and orphaned shared libaries to /usr/local/lib/compat/pkg updating symlinks properly. To keep binaries working, ldconfig(8) is run after each library deletion or move. Example: Phase preserving in portupgrade ---> Preserving /usr/local/lib/libdb-5.3.so.0 as /usr/local/lib/compat/pkg/libdb-5.3.so.0 cp -pf /usr/local/lib/libdb-5.3.so.0 /usr/local/lib/compat/pkg/libdb-5.3.so.0 It's really there at this moment: ls -l /usr/local/lib/compat/pkg/libdb-5.3.so.0 -rwxr-xr-x 1 root wheel 1652536 24 Apr 13:26 /usr/local/lib/compat/pkg/libdb-5.3.so.0 ... Phase Cleaning in portupgrade portsclean -L ** /usr/local/lib/compat/pkg/libdb-5.3.so.0 is shadowed by /usr/local/lib/libdb-5.3.so.0 --> The one in /usr/local/lib/compat/pkg is not used Delete /usr/local/lib/compat/pkg/libdb-5.3.so.0 So this change in portupgrade is not very useful, all (or most) preserved symlinks are immediately deleted and "Preserving ..." message is now very confusing.
Re-opening due to fallout from ports r463994 (originally reported in comment 12) and then in: Bug 237069 Bug 231694 Bug 227599
*** Bug 237069 has been marked as a duplicate of this bug. ***
> I don't understand what this has to do, -R with symbolic link doesn't make sense. In the cp(1) manpage: > -R ... This option also causes symbolic links to be copied, rather than indirected through It seems like a bug in cp(1) to me. How does someone copy a symlink onto an existing symlink? -f is supposed to avoid the error but it does not. GNU cp gets this right. ~/git/pkgtools % ls -al a total 10 drwxr-xr-x 2 bryan bryan 4 Oct 29 14:42 ./ drwxr-xr-x 13 bryan bryan 21 Oct 29 14:42 ../ -rw-r--r-- 1 bryan bryan 0 Oct 29 14:42 1 lrwxr-xr-x 1 bryan bryan 4 Oct 29 14:42 2@ -> blah ~/git/pkgtools % cp -R a b ~/git/pkgtools % ls -al b total 10 drwxr-xr-x 2 bryan bryan 4 Oct 29 14:46 ./ drwxr-xr-x 13 bryan bryan 21 Oct 29 14:46 ../ -rw-r--r-- 1 bryan bryan 0 Oct 29 14:46 1 lrwxr-xr-x 1 bryan bryan 4 Oct 29 14:46 2@ -> blah ~/git/pkgtools % find a \( -type f -o -type l \) -exec cp -vRfp {} b/ \; cp: symlink: blah: File exists a/1 -> b/1 And GNU cp: ~/git/pkgtools % find a \( -type f -o -type l \) -exec gcp -vRfp {} b/ \; removed 'b/2' 'a/2' -> 'b/2' 'a/1' -> 'b/1' ~/git/pkgtools % ls -al b total 10 drwxr-xr-x 2 bryan bryan 4 Oct 29 14:46 ./ drwxr-xr-x 13 bryan bryan 21 Oct 29 14:46 ../ -rw-r--r-- 1 bryan bryan 0 Oct 29 14:42 1 lrwxr-xr-x 1 bryan bryan 4 Oct 29 14:42 2@ -> blah Without -R it is wrong too as it tries to dereference a/2 to missing blah. ~/git/pkgtools % find a \( -type f -o -type l \) -exec gcp -vfp {} b/ \; gcp: cannot stat 'a/2': No such file or directory 'a/1' -> 'b/1'
To be clear the first example is FreeBSD cp.
(In reply to Bryan Drewery from comment #17) Also see bug 174489.