Bug 210313 - java/javavmwrapper fails with pkg upgrade -f
Summary: java/javavmwrapper fails with pkg upgrade -f
Status: Closed FIXED
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Some People
Assignee: Greg Lewis
URL:
Keywords:
: 212104 (view as bug list)
Depends on:
Blocks:
 
Reported: 2016-06-15 21:05 UTC by Kevin Bowling
Modified: 2016-09-08 12:16 UTC (History)
7 users (show)

See Also:
bugzilla: maintainer-feedback? (glewis)


Attachments
Change to link removal during post deinstall (364 bytes, patch)
2016-06-16 08:25 UTC, Greg Lewis
no flags Details | Diff
proposed change: @javavm + @sample (12.03 KB, patch)
2016-06-18 08:56 UTC, Baptiste Daroussin
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Kevin Bowling freebsd_committer 2016-06-15 21:05:16 UTC
When doing something like pkg upgrade -f or pkg install -f javavmwrapper, pkg exits with:
[1/1] Reinstalling javavmwrapper-2.5...
[1/1] Extracting javavmwrapper-2.5: 100%
pkg: Fail to rename /usr/local/bin/checkvms.OyXWrDokPDgu -> /usr/local/bin/checkvms: No such file or directory

Diagnosis from bapt:
[13:56] <bapt> yes there is a bug in the pkg-deinstall script in javawrapper
[13:57] <bapt> the script does rm -f /usr/local/bin/checkvms*
[13:57] <bapt> which breaks pkg upgrading
Comment 1 Baptiste Daroussin freebsd_committer 2016-06-15 21:08:48 UTC
I will have to workaround the issue in pkg anyway given that no users will ever be able to upgrade otherwise, still the pkg-deinstall script has to be fixed
Comment 2 Baptiste Daroussin freebsd_committer 2016-06-15 22:55:21 UTC
to be exact on deinstall it finds in everything symlinks to javavms in localbase/bin and nukes it

which is totally wrong anyway because it can nukes some users symlinks if any.

It should only nukes symlinks it knows it about and not all random.

because it is done in post deinstall it happens during an upgrade before pkg finishes its upgrade phase by renaming all the temporary files it has created (atomic upgrade at the level of a package) and thus removes pkg's temporary "symlinks" that will become the new checkvms, etc in the end
Comment 3 Greg Lewis freebsd_committer 2016-06-16 06:50:29 UTC
pkg-deinstall in javavmwrapper hasn't changed meaningfully since 2012 (the change in 2014 was SVN props only).

It is not accurate that the script does 'rm -f /usr/local/bin/checkvms*'.  Here is the script: https://svnweb.freebsd.org/ports/head/java/javavmwrapper/pkg-deinstall?view=markup

What it does is look for symlinks in the ${PKG_PREFIX}/bin which point to javavm that _aren't_ named checkvms, manvm, registervm, or unregistervm and deletes them (since it created them).

So it does only delete symlinks it knows about and doesn't pick random symlinks at all.

Now, based on having a file named 'checkvms.OyXWrDokPDgu', I'm guessing that pkg creates temporary named files somehow as part of its install/deinstall process?  Is that correct?

In terms of fixing this, if we wanted to only act during POST-DEINSTALL, then that would be a simple change to line 8 from checking for DEINSTALL to checking for POST-DEINSTALL.  Will that break the realpath check though?  Since the files javavmwrapper installs, including javavm, will be gone by then.  I'll need to check that to be sure.

If that works, it would also remove the need to exclude files like checkvms since they will already be gone.

That is the short term fix.  The long term fix is likely that javavmwrapper should be removed and the java ports should act more like other language ports that allow the user to select a default version and use that.
Comment 4 Greg Lewis freebsd_committer 2016-06-16 08:25:28 UTC
Created attachment 171475 [details]
Change to link removal during post deinstall

If all we need to do is to switch to doing this during post deinstall then the attached patch should work.  I'll test it as soon as I can.
Comment 5 Baptiste Daroussin freebsd_committer 2016-06-16 09:39:25 UTC
that is imho just workarounding the problem

removeing all symlinks to javavms in bin is still wrong imho considering a user can have created its own symlinks for various reasons and they would be removed.

My question is why does the deinstall command try to remove all symlinks to javavms? it javavmwrapper creating them? if yes it should be doable to track which one it has created through a strict pattern or a cache file that keeps track of it?
Comment 6 Greg Lewis freebsd_committer 2016-06-18 05:52:47 UTC
Yes, creating symlinks is one of the main things javavmwrapper does.  All the JDK ports install in their own separate directories.  javavmwrapper creates links in /usr/local/bin to all their executables (e.g. /usr/local/bin/java).  So yeah, when it is being deinstalled it removes those.

Could it cache them?  Probably?  Doing that kind of thing in a shell script doesn't seem immediately intuitive, since it has to track which links belong to which JDK install since they can differ across versions in their executables.  I'll think about it some more.

Could the user have created their own symlink?  They could have, but it wouldn't work.  When the symlink is executed javavmwrapper tries to match it to an executable in an installed jdk.  The user making an arbitrary symlink that doesn't correspond to anything in the jdk is just going to error.
Comment 7 Baptiste Daroussin freebsd_committer 2016-06-18 08:56:07 UTC
Created attachment 171537 [details]
proposed change: @javavm + @sample
Comment 8 Baptiste Daroussin freebsd_committer 2016-06-18 08:56:34 UTC
from what I read in the code unregistervm is doing the proper thing of removing only the required symlinks for a given jdk. So I do not see the point removing them in a javawmwrapper script.

All jdk/jre are running the registervm/unregistervm properly

What I do propose it to entirely remove the post and pre script (replacing the configuration file handling by the @sample keyword)

Add a new @javawm keyword that will add the unregistervm/registervm properly (always post-install for registervm and pre-deinstall for unregistervm)

Make all jdk/jre ports using that new keyword

Note that when creating the keyword I have not made considering the argument file as a "regular file" for plist because all openjdk ports have a dynamically generated plist that will always add */bin/java to plist.

So for now the semantic in plist is:
XXX/bin/java
@javawm XXX/bin/java

if one has to delete everything deleting the jdk packages will (and already does) properly remove all symlinks. I do no see a point in removing those symlinks in javavmwrapper package itself.

What do you think? am I missing something?

Note that if one day we end up changing the javavmwrapper behaviour all the javavmwrapper would have to do is running checkvms in post-install
Comment 9 Greg Lewis freebsd_committer 2016-06-20 06:48:24 UTC
I've looked over the patch.  I'm not clear what happens when one deinstalls javavmwrapper and/or upgrades it.  Will the user be left with a lot of dangling symlinks if its a deinstall?
Comment 10 Baptiste Daroussin freebsd_committer 2016-06-20 06:52:42 UTC
javavmwrapper does not handle anymore the symlinks during the install phase/deinstall phase.

The symlinks are created and removed by registervm/unregistervm, meaning they are created/removed when a java package is added/removed.

Given the java packages always depends on javavmwrapper I do think that is good enough as to deinstall javavmwrapper one has to first remove the jdk java packages which will result in removing the symlinks, then delete javavmwrapper.

That's why I am asking if I am missing something I could not find anything else that creates symlinks then registervm
Comment 11 Baptiste Daroussin freebsd_committer 2016-06-27 06:17:26 UTC
any update here?
Comment 12 Andres Montalban 2016-06-27 16:20:11 UTC
+1
Comment 13 commit-hook freebsd_committer 2016-07-31 11:53:12 UTC
A commit references this bug:

Author: bapt
Date: Sun Jul 31 11:52:23 UTC 2016
New revision: 419360
URL: https://svnweb.freebsd.org/changeset/ports/419360

Log:
  Add a hack in pkg to skip running predeinstall for javavmwrapper-2.5

  During upgrades/reinstall javavmwrapper 2.5 pre deinstall script deletes the pkg
  temporary files preventing to finish the upgrade.

  A fixed version of javavmwrapper will be made soon, this hack will remain in the
  ports tree and not in pkg(8) itself. This hack is made to not bother users is
  only affecting upgrade/reinstall phase not proper deinstall

  PR:		210313

Changes:
  head/ports-mgmt/pkg/Makefile
  head/ports-mgmt/pkg/files/patch-javavmwrapper
Comment 14 commit-hook freebsd_committer 2016-07-31 11:55:15 UTC
A commit references this bug:

Author: bapt
Date: Sun Jul 31 11:54:32 UTC 2016
New revision: 419361
URL: https://svnweb.freebsd.org/changeset/ports/419361

Log:
  MFH: r419360

  Add a hack in pkg to skip running predeinstall for javavmwrapper-2.5

  During upgrades/reinstall javavmwrapper 2.5 pre deinstall script deletes the pkg
  temporary files preventing to finish the upgrade.

  A fixed version of javavmwrapper will be made soon, this hack will remain in the
  ports tree and not in pkg(8) itself. This hack is made to not bother users is
  only affecting upgrade/reinstall phase not proper deinstall

  PR:		210313

Changes:
_U  branches/2016Q3/
  branches/2016Q3/ports-mgmt/pkg/Makefile
  branches/2016Q3/ports-mgmt/pkg/files/patch-javavmwrapper
Comment 15 Baptiste Daroussin freebsd_committer 2016-07-31 12:13:55 UTC
I am going to commit the javavmwrapper thing now, because I also found another issue due to the deinstall scripts: if one goes in the javavmwrapper ports directory and run
make deinstall reinstall
it will nuke all the links to any java VM even if there are some installed
Comment 16 Baptiste Daroussin freebsd_committer 2016-07-31 12:16:45 UTC
Forget that last comment it is wrong, bad testing env
Comment 17 commit-hook freebsd_committer 2016-07-31 12:31:22 UTC
A commit references this bug:

Author: bapt
Date: Sun Jul 31 12:30:24 UTC 2016
New revision: 419364
URL: https://svnweb.freebsd.org/changeset/ports/419364

Log:
  Remove the deinstall script that appears to be deleting all the temporary files
  created by pkg(8) during upgrades

  It happens because the deinstall script tries to clean up the potential manual
  VM registration by cleaning out all symlinks to bin/javavm

  Given all VM are registring/unregistering themselves this part is not needed

  The other thing the script was doing handling the configuration which has been
  replaced by @sample.

  pkg-install has been modified to drop the handling of the configuration file but
  keep the auto registration if all VM found. While this part is not necessary as
  well, we keep it because otherwise anyone doing delete/install on javavmwapper
  version 2.5 being the installed version would end up with all VM unregistered.

  The pkg-install should be removed after EOL of FreeBSD 10.3

  PR:		210313
  MFH:		2016Q3

Changes:
  head/java/javavmwrapper/Makefile
  head/java/javavmwrapper/files/pkg-install.in
  head/java/javavmwrapper/pkg-plist
Comment 18 commit-hook freebsd_committer 2016-07-31 12:32:24 UTC
A commit references this bug:

Author: bapt
Date: Sun Jul 31 12:31:53 UTC 2016
New revision: 419365
URL: https://svnweb.freebsd.org/changeset/ports/419365

Log:
  MFH: r419364

  Remove the deinstall script that appears to be deleting all the temporary files
  created by pkg(8) during upgrades

  It happens because the deinstall script tries to clean up the potential manual
  VM registration by cleaning out all symlinks to bin/javavm

  Given all VM are registring/unregistering themselves this part is not needed

  The other thing the script was doing handling the configuration which has been
  replaced by @sample.

  pkg-install has been modified to drop the handling of the configuration file but
  keep the auto registration if all VM found. While this part is not necessary as
  well, we keep it because otherwise anyone doing delete/install on javavmwapper
  version 2.5 being the installed version would end up with all VM unregistered.

  The pkg-install should be removed after EOL of FreeBSD 10.3

  PR:		210313

Changes:
_U  branches/2016Q3/
  branches/2016Q3/java/javavmwrapper/Makefile
  branches/2016Q3/java/javavmwrapper/files/pkg-install.in
  branches/2016Q3/java/javavmwrapper/pkg-plist
Comment 19 Andres Montalban 2016-08-05 19:04:35 UTC
(In reply to commit-hook from comment #18)
This seems to be not fixed for me:

root@ip-10-0-1-34:~ # pkg upgrade -f -y javavmwrapper
Updating FreeBSD repository catalogue...
FreeBSD repository is up-to-date.
All repositories are up-to-date.
Checking integrity... done (0 conflicting)
The following 1 package(s) will be affected (of 0 checked):

Installed packages to be REINSTALLED:
        javavmwrapper-2.5_1

Number of packages to be reinstalled: 1
[1/1] Reinstalling javavmwrapper-2.5_1...
[1/1] Extracting javavmwrapper-2.5_1: 100%
You may need to manually remove /usr/local/etc/javavm_opts.conf if it is no longer needed.
pkg: Fail to rename /usr/local/bin/.checkvms.61GPGU90MWCB -> /usr/local/bin/checkvms: No such file or directory

Tried several times and fails.

Thanks!
Comment 20 commit-hook freebsd_committer 2016-08-23 12:29:08 UTC
A commit references this bug:

Author: bapt
Date: Tue Aug 23 12:28:07 UTC 2016
New revision: 420669
URL: https://svnweb.freebsd.org/changeset/ports/420669

Log:
  Remove the pkg-deinstall script forgotten during r419364

  Pointyhat:	bapt
  PR:		210313

Changes:
  head/java/javavmwrapper/Makefile
  head/java/javavmwrapper/pkg-deinstall
Comment 21 rhs 2016-08-27 11:24:37 UTC
Tried to upgrade system to use port version of ssh ie 'security/openssh'. I have been using 'synth' for ports building for about 6mnths now and it has been spot on except for this error:

Fail to rename /usr/local/bin/.checkvms.QcpPjgu4LQDu -> /usr/local/bin/checkvms: No such file or directory
Unfortunately, the system upgrade failed.

I checked the makefile and it was ports revision 2 so I guess that's the latest?

I am confused here, i.e. has the hack been committed or just proposed?
Comment 22 Baptiste Daroussin freebsd_committer 2016-09-08 11:46:31 UTC
*** Bug 212104 has been marked as a duplicate of this bug. ***
Comment 23 Baptiste Daroussin freebsd_committer 2016-09-08 11:48:36 UTC
the _2 is the first one properly fixed. but if you upgrade from an older version then during the upgrade the deinstall script from the old version is used not the new one

the day we have a _3 and you upgrade from _2 to _3 this issue will happen
Comment 24 Kubilay Kocak freebsd_committer freebsd_triage 2016-09-08 12:15:22 UTC
(In reply to Baptiste Daroussin from comment #23)

So is this issue open (not completely/permanently resolved) or closed?
Comment 25 Baptiste Daroussin freebsd_committer 2016-09-08 12:16:53 UTC
it is closed as there is nothing more we can do there :)