Bug 232501 - lang/perl5.26: $^X is non-deterministic
Summary: lang/perl5.26: $^X is non-deterministic
Status: Closed FIXED
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: amd64 Any
: --- Affects Many People
Assignee: Mathieu Arnold
URL:
Keywords:
Depends on:
Blocks: 232146
  Show dependency treegraph
 
Reported: 2018-10-21 10:13 UTC by Helge Oldach
Modified: 2018-11-01 16:33 UTC (History)
1 user (show)

See Also:
bugzilla: maintainer-feedback? (mat)


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Helge Oldach 2018-10-21 10:13:26 UTC
This popped up in bug #232146. perl, perl5 and perl5.26.2 are hard links to the same image. When asking $^X ("the name by which this perl interpreter was invoked") it appears that the output depends on which of the three image names had been used recently. All potential invocations of perl, perl5, or perl5.26.2 appear to show up over time as $^X.

The behavior of $^X actually depends on which of the three hard links had been used /last/ to run perl. Once all three had been called, the last one remains sticky. Here's a quick test:

# reboot straight into single-user, don't run anything 
# perl -e 'print $^X, "\n"'
/usr/local/bin/perl
# perl5 -e 'print $^X, "\n"'
/usr/local/bin/perl5
# perl5.26.2 -e 'print $^X, "\n"'
/usr/local/bin/perl5.26.2
# perl -e 'print $^X, "\n"'
/usr/local/bin/perl5.26.2
# perl5 -e 'print $^X, "\n"'
/usr/local/bin/perl5.26.2
# perl5.26.2 -e 'print $^X, "\n"'
/usr/local/bin/perl5.26.2
# perl -e 'print $^X, "\n"'
/usr/local/bin/perl5.26.2
# perl5 -e 'print $^X, "\n"'
/usr/local/bin/perl5.26.2
# perl5.26.2 -e 'print $^X, "\n"'
/usr/local/bin/perl5.26.2

# reboot straight into single-user, don't run anything 
# perl5.26.2 -e 'print $^X, "\n"'
/usr/local/bin/perl5.26.2
# perl5 -e 'print $^X, "\n"'
/usr/local/bin/perl5
# perl -e 'print $^X, "\n"'
/usr/local/bin/perl
# perl5.26.2 -e 'print $^X, "\n"'
/usr/local/bin/perl
# perl5 -e 'print $^X, "\n"'
/usr/local/bin/perl
# perl -e 'print $^X, "\n"'
/usr/local/bin/perl
# perl5.26.2 -e 'print $^X, "\n"'
/usr/local/bin/perl
# perl5 -e 'print $^X, "\n"'
/usr/local/bin/perl
# perl -e 'print $^X, "\n"'
/usr/local/bin/perl

# reboot straight into single-user, don't run anything 
# perl -e 'print $^X, "\n"'
/usr/local/bin/perl
# perl5.26.2 -e 'print $^X, "\n"'
/usr/local/bin/perl5.26.2
# perl5 -e 'print $^X, "\n"'
/usr/local/bin/perl5
# perl -e 'print $^X, "\n"'
/usr/local/bin/perl5
# perl5.26.2 -e 'print $^X, "\n"'
/usr/local/bin/perl5
# perl5 -e 'print $^X, "\n"'
/usr/local/bin/perl5
# perl -e 'print $^X, "\n"'
/usr/local/bin/perl5
# perl5.26.2 -e 'print $^X, "\n"'
/usr/local/bin/perl5
# perl5 -e 'print $^X, "\n"'
/usr/local/bin/perl5

In other words, use of $^X is non-deterministic in FreeBSD.

I suspect the canonical fix would be to replace the perl/perl5/perl5.26.2 hard links with soft links perl -> perl5 -> perl5.26.2?

This is seen on FreeBSD 11.2-STABLE r339287 amd64.
Comment 1 Mathieu Arnold freebsd_committer 2018-10-25 14:30:20 UTC
The thing is, nothing should ever use anything else than `perl`.

The perl5 link is probably never used and some old heritage that predates me.

The versionned link should never also never be used directly, there is a qa check to make sure no shebang points to it.  It should never be used because if it is, then pkg will fail to restart the services when Perl's version change during an upgrade, which is close to the problem you are having in #232146.
Comment 2 Mathieu Arnold freebsd_committer 2018-10-25 14:54:31 UTC
Though, I can't say I can reproduce your claim

root@11amd64-ports:~ # perl5 -E 'say $^X'
/usr/local/bin/perl5.26.2
root@11amd64-ports:~ # perl5.26.2  -E 'say $^X'
/usr/local/bin/perl5.26.2
root@11amd64-ports:~ # perl -E 'say $^X'
/usr/local/bin/perl5.26.2
root@11amd64-ports:~ # logout
Comment 3 Mathieu Arnold freebsd_committer 2018-10-25 14:58:21 UTC
maybe it is a filesystem thing, I don't have any UFS machine to test on.
Comment 4 Helge Oldach 2018-10-25 15:05:25 UTC
Agreed, but the versioned link is there, and the perl5 link as well. Both are installed by lang/perl5.26. Hence they will be used sometime by somebody, even if all our ports are QA-cleaned (specifying just "perl" in the shebang). And when somebody has used it, it's sticky as in the example above. From that point onward our ports will break to reload or restart... bug #232146 for example.

Actually other ports tackle this differently. For instance, certbot (the Letsencrypt client) has /usr/local/bin/pyhon2.7 in the shebang - that is the versioned name. But there is no /usr/local/bin/python unless lang/python is installed as well on top of lang/python27. And if, it's symlinked (not hardlinked like perl): python -> python2 -> python2.7  - which then will do the right thing.

Why don't the perl ports follow the same logic?
Comment 5 Helge Oldach 2018-10-25 15:06:27 UTC
(In reply to Mathieu Arnold from comment #3)
Please note I have rebooted between the perl incancations into single-user in order to have a clean environment.
Comment 6 Mathieu Arnold freebsd_committer 2018-10-25 15:23:03 UTC
Could you try this patch https://github.com/mat813/freebsd-ports/commit/2e284c7ecc14b737d6dda506b7c2419017d55f54 and report back?
Comment 7 Mathieu Arnold freebsd_committer 2018-10-25 15:23:54 UTC
because it is not perl5.26, it is perl5.26.2, when it goes to 5.26.3, none of the services could be restarted.
Comment 8 Helge Oldach 2018-10-25 15:28:45 UTC
(In reply to Helge Oldach from comment #4)
For the sake of the exercise I did a quick test, mimicking the python logic of symlinking towards the actual image:

# cd /usr/local/bin
# rm perl; ln -sf perl5 perl
# rm perl5; ln -sf perl5.26.2  perl5
lrwxr-xr-x  1 root  wheel       5 Oct 25 17:17 perl@ -> perl5
lrwxr-xr-x  1 root  wheel      10 Oct 25 17:17 perl5@ -> perl5.26.2
-rwxr-xr-x  1 root  wheel    7792 Apr 19  2018 perl5.26.2*
-rwxr-xr-x  2 root  wheel   45866 Apr 19  2018 perlbug*
...
#

# reboot into single user in order to have a clean environment
# perl -e 'print $^X, "\n"'
/usr/local/bin/perl5.26.2
# perl5 -e 'print $^X, "\n"'
/usr/local/bin/perl5.26.2
# perl5.26.2 -e 'print $^X, "\n"'
/usr/local/bin/perl5.26.2
#

As can be clearly seen, $^X is deterministic and always pointing to the true image name. Hence it should be good enough to be included in shebang lines.
Comment 9 Helge Oldach 2018-10-25 15:35:25 UTC
(In reply to Mathieu Arnold from comment #7)
That will do, but here the logic of the symlink chain is reversed compared to libperl.so (visible in the patch). Kind of POLA violation?

IMHO the more logical approach would be to follow the python logic perl-> perl5 -> perl5.26.2.

Then, of course, the shebang of all dependent ports must be corrected...
Comment 10 Mathieu Arnold freebsd_committer 2018-10-31 15:20:54 UTC
So, does my change fix the issue for you?
Comment 11 Helge Oldach 2018-10-31 16:08:34 UTC
(In reply to Mathieu Arnold from comment #10)
Technically it does fix it indeed, but again, I believe the symlinks should be in the reverse order perl -> perl5 -> perl5.26.2 - following the libperl.so symlink sequence.
Comment 12 Mathieu Arnold freebsd_committer 2018-11-01 06:59:34 UTC
I do not understand, have you tested the change, or not?  If not, can you test it and please report back.
Comment 13 Helge Oldach 2018-11-01 07:40:39 UTC
(In reply to Mathieu Arnold from comment #12)
Once again, the patch works. But it is wrong IMHO. It should be perl -> perl5 -> perl5.26.2, but you have implemented perl5.26.2 -> perl and perl5 -> perl.
Comment 14 Mathieu Arnold freebsd_committer 2018-11-01 10:25:57 UTC
As I already explained a few times, it cannot be what you say.  Thank you for reporting back.
Comment 15 commit-hook freebsd_committer 2018-11-01 10:37:32 UTC
A commit references this bug:

Author: mat
Date: Thu Nov  1 10:36:33 UTC 2018
New revision: 483655
URL: https://svnweb.freebsd.org/changeset/ports/483655

Log:
  Make $^X is deterministic.

  Its output can vary, and not be the correct one because of hard links.
  So replace the hard links with symlinks so that there is only one perl.

  PR:		232501
  Reported by:	Helge Oldach

Changes:
  head/lang/perl5-devel/Makefile
  head/lang/perl5.24/Makefile
  head/lang/perl5.26/Makefile
  head/lang/perl5.28/Makefile