Bug 216346 - emulators/linux_base-c6: Mathematica 10.x doesn't work with recent linux_base-c6 or -c7
Summary: emulators/linux_base-c6: Mathematica 10.x doesn't work with recent linux_base...
Status: Closed FIXED
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: amd64 Any
: --- Affects Many People
Assignee: Tijl Coosemans
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2017-01-21 16:58 UTC by Osamu Matsuda
Modified: 2017-02-08 03:21 UTC (History)
2 users (show)

See Also:
bugzilla: maintainer-feedback? (emulation)


Attachments
ktrace output (681.05 KB, application/x-xz)
2017-01-23 15:55 UTC, Osamu Matsuda
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Osamu Matsuda 2017-01-21 16:58:15 UTC
Hi.
I've been using Mathematica 10.x for years successfully
with FreeBSD, but recently it just freezes at the initialization stage
with a message "initializing kernels..." on the initial
Mathematica logo.  At that time, I see a message as follows.

% mathematica
/compat/linux/bin/sh: lfram/Mathematica/10.0/Executables/math: No such file or directory
/compat/linux/bin/sh: lfram/Mathematica/10.0/Executables/math: No such file or directory

The actual running processes are as follows.

% ps ww
 PID TT  STAT    TIME COMMAND
2547  0  I    0:00.00 /bin/sh /usr/local/bin/mathematica
2612  0  S    0:00.86 /usr/local/Wolfram/Mathematica/10.0/SystemFiles/FrontEnd/Binaries/Linux/Mathematica -topDirectory /usr/local/Wolfram/Mathematica/10.0


It seems the process 2612 tries to launch
"/usr/local/Wolfram/Mathematica/10.0/Executables/math",
but it is erroneously looking for the file 
"lfram/Mathematica/10.0/Executables/math",
and failed.

I have tried several combinations of the OSes, mathematica versions,
and linux_base versions as follows.

OS (amd64):
FreeBSD 10.3-RELEASE     
FreeBSD 10.3-RELEASE-p16
FreeBSD 11.0-RELEASE-p7

Mathematica: (32bit version)
mathematica 10.0 
mathematica 10.2

linux_base
linux_base-c7-7.3.1611         freeze
linux_base-c6-6.8_9 (r431790)  freeze
linux_base-c6-6.8_8 (r431465)  freeze
linux_base-c6-6.8_1 (r421307)  freeze
linux_base-c6-6.8   (r419460)  OK
linux_base-c6-6.7_1 (r409060)  OK


The result only depends on the version of linux_base.  It
seems some modification made between linux_base-c6-6.8 and
linux_base-c6-6.8_1 would be responsible for this regression.  

I would be happy if someone helps me to find out which file installed
by linux_base causes this problem.
Comment 1 Tijl Coosemans freebsd_committer freebsd_triage 2017-01-21 17:46:06 UTC
What is inside /usr/local/bin/mathematica?
What is the output of "find /compat/linux/dev"?
Is linux_base the only dependency or do you have other linux packages installed?
Do you have linprocfs mounted on /compat/linux/proc?
Do you have tmpfs mounted on /compat/linux/dev/shm?
What is the output of "sysctl compat"?
What is the output of "ls -l /usr/local/Wolfram/Mathematica/10.0/Executables/math"?
Comment 2 Osamu Matsuda 2017-01-22 14:42:04 UTC
Below is the required information for the case of using mathematica 10.0.


> What is inside /usr/local/bin/mathematica?

/usr/local/bin/mathematica is linked to
/usr/local/Wolfram/Mathematica/10.0/Executables/mathematica.

In the directory /usr/local/Wolfram/Mathematica/10.0/Executables/,
there are 7 shell scripts, namely MathKernel, Mathematica,
WolframKernel, math mathematica, mcc, and wolfram.  Each of them needs
to be patched according to the FreeBSD handbook.  Below I put the diff
of them.

-------- begin -----------------------

*** MathKernel.orig	Tue Oct 21 23:40:32 2014
--- MathKernel	Wed Oct 22 00:13:27 2014
***************
*** 5,10 ****
--- 5,11 ----
  
  #  Make certain that ${PATH} includes /usr/bin and /bin
  PATH="/usr/bin:/bin:${PATH}"
+ SystemIDList=Linux
  
  # Is systemid being passed on the command-line?
  case "$1" in
***************
*** 50,56 ****
  #  the output of ls -l on the symlink to find the link target. Reset
  #  ${Script}.
  LinkDirectory=`pwd`
! while `exec test -L "${Script}"`; do
          ScriptDirectory=`dirname "${Script}"`
          Script=`ls -l "${Script}" | sed -e 's/.*-> //g'`
          cd "${ScriptDirectory}"
--- 51,57 ----
  #  the output of ls -l on the symlink to find the link target. Reset
  #  ${Script}.
  LinkDirectory=`pwd`
! while `test -L "${Script}"`; do
          ScriptDirectory=`dirname "${Script}"`
          Script=`ls -l "${Script}" | sed -e 's/.*-> //g'`
          cd "${ScriptDirectory}"
***************
*** 132,138 ****
  
      if [ "${SystemID}" = "Linux" -o "${SystemID}" = "Linux-x86-64" ]; then
          if [ -z "${OMP_NUM_THREADS}" ]; then
!             OMP_NUM_THREADS=`cat /proc/cpuinfo | grep '^processor.: [0-9]*$' | wc -l | tr -d ' '`;
              export OMP_NUM_THREADS;
          fi
      fi
--- 133,139 ----
  
      if [ "${SystemID}" = "Linux" -o "${SystemID}" = "Linux-x86-64" ]; then
          if [ -z "${OMP_NUM_THREADS}" ]; then
!             OMP_NUM_THREADS=`cat /compat/linux/proc/cpuinfo | grep '^processor.: [0-9]*$' | wc -l | tr -d ' '`;
              export OMP_NUM_THREADS;
          fi
      fi



*** Mathematica.orig	Tue Oct 21 23:40:44 2014
--- Mathematica	Wed Oct 22 00:15:20 2014
***************
*** 5,11 ****
  
  #  Make certain that ${PATH} includes /usr/bin and /bin
  PATH="/usr/bin:/bin:${PATH}"
! 
  
  
  
--- 5,11 ----
  
  #  Make certain that ${PATH} includes /usr/bin and /bin
  PATH="/usr/bin:/bin:${PATH}"
! SystemIDList=Linux
  
  
  
***************
*** 46,52 ****
  #  the output of ls -l on the symlink to find the link target. Reset
  #  ${Script}.
  LinkDirectory=`pwd`
! while `exec test -L "${Script}"`; do
  	ScriptDirectory=`dirname "${Script}"`
  	Script=`ls -l "${Script}" | sed -e 's/.*-> //g'`
  	cd "${ScriptDirectory}"
--- 46,52 ----
  #  the output of ls -l on the symlink to find the link target. Reset
  #  ${Script}.
  LinkDirectory=`pwd`
! while `test -L "${Script}"`; do
  	ScriptDirectory=`dirname "${Script}"`
  	Script=`ls -l "${Script}" | sed -e 's/.*-> //g'`
  	cd "${ScriptDirectory}"



*** math.orig	Tue Oct 21 23:40:32 2014
--- math	Wed Oct 22 00:16:33 2014
***************
*** 5,10 ****
--- 5,11 ----
  
  #  Make certain that ${PATH} includes /usr/bin and /bin
  PATH="/usr/bin:/bin:${PATH}"
+ SystemIDList=Linux
  
  # Is systemid being passed on the command-line?
  case "$1" in
***************
*** 50,56 ****
  #  the output of ls -l on the symlink to find the link target. Reset
  #  ${Script}.
  LinkDirectory=`pwd`
! while `exec test -L "${Script}"`; do
          ScriptDirectory=`dirname "${Script}"`
          Script=`ls -l "${Script}" | sed -e 's/.*-> //g'`
          cd "${ScriptDirectory}"
--- 51,57 ----
  #  the output of ls -l on the symlink to find the link target. Reset
  #  ${Script}.
  LinkDirectory=`pwd`
! while `test -L "${Script}"`; do
          ScriptDirectory=`dirname "${Script}"`
          Script=`ls -l "${Script}" | sed -e 's/.*-> //g'`
          cd "${ScriptDirectory}"
***************
*** 132,138 ****
  
      if [ "${SystemID}" = "Linux" -o "${SystemID}" = "Linux-x86-64" ]; then
          if [ -z "${OMP_NUM_THREADS}" ]; then
!             OMP_NUM_THREADS=`cat /proc/cpuinfo | grep '^processor.: [0-9]*$' | wc -l | tr -d ' '`;
              export OMP_NUM_THREADS;
          fi
      fi
--- 133,139 ----
  
      if [ "${SystemID}" = "Linux" -o "${SystemID}" = "Linux-x86-64" ]; then
          if [ -z "${OMP_NUM_THREADS}" ]; then
!             OMP_NUM_THREADS=`cat /compat/linux/proc/cpuinfo | grep '^processor.: [0-9]*$' | wc -l | tr -d ' '`;
              export OMP_NUM_THREADS;
          fi
      fi



*** mathematica.orig	Tue Oct 21 23:40:44 2014
--- mathematica	Wed Oct 22 00:18:33 2014
***************
*** 5,11 ****
  
  #  Make certain that ${PATH} includes /usr/bin and /bin
  PATH="/usr/bin:/bin:${PATH}"
! 
  
  
  
--- 5,11 ----
  
  #  Make certain that ${PATH} includes /usr/bin and /bin
  PATH="/usr/bin:/bin:${PATH}"
! SystemIDList=Linux
  
  
  
***************
*** 46,52 ****
  #  the output of ls -l on the symlink to find the link target. Reset
  #  ${Script}.
  LinkDirectory=`pwd`
! while `exec test -L "${Script}"`; do
  	ScriptDirectory=`dirname "${Script}"`
  	Script=`ls -l "${Script}" | sed -e 's/.*-> //g'`
  	cd "${ScriptDirectory}"
--- 46,52 ----
  #  the output of ls -l on the symlink to find the link target. Reset
  #  ${Script}.
  LinkDirectory=`pwd`
! while `test -L "${Script}"`; do
  	ScriptDirectory=`dirname "${Script}"`
  	Script=`ls -l "${Script}" | sed -e 's/.*-> //g'`
  	cd "${ScriptDirectory}"



*** mcc.orig	Tue Oct 21 23:41:07 2014
--- mcc	Wed Oct 22 00:19:08 2014
***************
*** 5,10 ****
--- 5,11 ----
  
  #  Make certain that ${PATH} includes /usr/bin and /bin
  PATH="/usr/bin:/bin:${PATH}"
+ SystemIDList=Linux
  
  #  Determine the SystemID by examining the output of `uname -s` and 
  #  `uname -m`. Failsafe to SystemIDList=Unknown.
***************
*** 57,63 ****
  #  the output of ls -l on the symlink to find the link target. Reset
  #  ${Script}.
  LinkDirectory=`pwd`
! while `exec test -L "${Script}"`; do
          ScriptDirectory=`dirname "${Script}"`
          Script=`ls -l "${Script}" | sed -e 's/.*-> //g'`
          cd "${ScriptDirectory}"
--- 58,64 ----
  #  the output of ls -l on the symlink to find the link target. Reset
  #  ${Script}.
  LinkDirectory=`pwd`
! while `test -L "${Script}"`; do
          ScriptDirectory=`dirname "${Script}"`
          Script=`ls -l "${Script}" | sed -e 's/.*-> //g'`
          cd "${ScriptDirectory}"


-------- end -------------------


It might also be necessary to ensure that /usr/local/bin/MathematicaScript is
linked to /usr/local/Wolfram/Mathematica/10.0/SystemFiles/Kernel/Binaries/Linux/MathematicaScript



> What is the output of "find /compat/linux/dev"?

~ 1007 % find /compat/linux/dev
/compat/linux/dev
/compat/linux/dev/shm
/compat/linux/dev/.shm.bJOs6e5BTk10


> Is linux_base the only dependency or do you have other linux packages installed?

I also installed emulators/linux-c6 or emulators/linux-c7 metaport
which also has installed a bunch of linux related ports.

> Do you have linprocfs mounted on /compat/linux/proc?
> Do you have tmpfs mounted on /compat/linux/dev/shm?

Yes for both.

~ 1003 % mount
/dev/mirror/root on / (ufs, NFS exported, local, journaled soft-updates)
devfs on /dev (devfs, local, multilabel)
procfs on /proc (procfs, local)
fdescfs on /dev/fd (fdescfs)
linprocfs on /compat/linux/proc (linprocfs, local)
tmpfs on /compat/linux/dev/shm (tmpfs, local)



> What is the output of "sysctl compat"?

~ 1005 % sysctl compat
compat.linux32.maxvmem: 0
compat.linux32.maxssiz: 67108864
compat.linux32.maxdsiz: 536870912
compat.linux.oss_version: 198144
compat.linux.osrelease: 2.6.18
compat.linux.osname: Linux
compat.ia32.maxvmem: 0
compat.ia32.maxssiz: 67108864
compat.ia32.maxdsiz: 536870912



> What is the output of "ls -l /usr/local/Wolfram/Mathematica/10.0/Executables/math"?

~ 1006 % ls -l /usr/local/Wolfram/Mathematica/10.0/Executables/math
-rwxr-xr-x 1 root wheel 5311 Aug 30 02:53 /usr/local/Wolfram/Mathematica/10.0/Executables/math*

As mentioned above, this is the patched version.
Comment 3 Tijl Coosemans freebsd_committer freebsd_triage 2017-01-22 15:11:56 UTC
(In reply to Osamu Matsuda from comment #2)
Try removing all those changes you made to the scripts and replace /usr/local/bin/mathematica with a startup script with the following content:

#! /bin/sh
exec /compat/linux/bin/sh /usr/local/Wolfram/Mathematica/10.0/Executables/mathematica

By running with linux sh the changes should not be necessary.

Because compat.linux.osrelease is 2.6.18 I suspect you set this in /etc/sysctl.conf.  Please remove that.  The default value should be 2.6.32.

Also remove /compat/linux/dev/.shm.bJOs6e5BTk10.  There should be nothing in that directory except the shm subdirectory.
Comment 4 Osamu Matsuda 2017-01-23 07:36:22 UTC
Thank you for your suggestion.

> Try removing all those changes you made to the scripts and replace /usr/local/bin/mathematica with a startup script with the following content:

> #! /bin/sh
> exec /compat/linux/bin/sh /usr/local/Wolfram/Mathematica/10.0/Executables/mathematica

> By running with linux sh the changes should not be necessary.

I have reinstalled Mathematica, and made the suggested replacement for
/usr/local/bin/mathematica as well as the modification to the link
/usr/local/bin/MathematicaScript, on my system with FreeBSD
10.3-RELEASE-p11, Mathematica 10.0, and linux_base-c7-7.3.1611.
Unfortunately, I've got the same result: it freezes with the
message as before.

  /compat/linux/bin/sh: lfram/Mathematica/10.0/Executables/math: No such file or directory
  /compat/linux/bin/sh: lfram/Mathematica/10.0/Executables/math: No such file or directory



(With linux_base-c6-6.8_9, I've just got the following message.

  mathematica cannot determine operating system.

It seems we still have to modify the shell scripts in 
/usr/local/Wolfram/Mathematica/10.0/Executables/*
for linux_base-c6.)



> Because compat.linux.osrelease is 2.6.18 I suspect you set this in
> /etc/sysctl.conf.  Please remove that.  The default value should be
> 2.6.32. 

> Also remove /compat/linux/dev/.shm.bJOs6e5BTk10.  There should be
> nothing in that directory except the shm subdirectory.

Thank you.  I have corrected these points, but the result is the same.
Comment 5 Tijl Coosemans freebsd_committer freebsd_triage 2017-01-23 11:08:16 UTC
Please run these two commands and attach kdump.txt.xz.  If it's too big you can email it to me directly or upload it somewhere and email me the link.

ktrace -f /tmp/ktrace.out -i mathematica

kill mathematica (e.g. with ctrl+C)

kdump -f /tmp/ktrace.out -H | xz > /tmp/kdump.txt.xz
Comment 6 Osamu Matsuda 2017-01-23 15:55:26 UTC
Created attachment 179259 [details]
ktrace output

Thank you very much for spending your time on this issue.
I attach the ktrace output to this.
Comment 7 Tijl Coosemans freebsd_committer freebsd_triage 2017-01-23 22:14:01 UTC
A possible workaround is to modify /usr/local/Wolfram/Mathematica/10.0/Executables/math.  Change "/bin/sh" on the first line into "/compat/linux/bin/sh".

To fix the "cannot determine operating system" error on linux_base-c6 create the following symlink: ln -s /bin/uname /compat/linux/usr/bin/uname
An alternative is to comment out line 7 PATH="/usr/bin:/bin:${PATH}" in /usr/local/Wolfram/Mathematica/10.0/SystemFiles/FrontEnd/Binaries/Linux/Mathematica.  By putting /usr/bin before /bin the script runs FreeBSD /usr/bin/uname instead of /compat/linux/bin/uname.  This problem doesn't exist with linux_base-c7 because /compat/linux/usr/bin/uname exists there.
Comment 8 Thomas Mueller 2017-01-24 15:08:43 UTC
(In reply to Osamu Matsuda from comment #0)
>% mathematica
>/compat/linux/bin/sh: lfram/Mathematica/10.0/Executables/math: No such file or directory
>/compat/linux/bin/sh: lfram/Mathematica/10.0/Executables/math: No such file or directory

The clobbered pathnames look familiar. I had similar issues with a product
developed @work with linux-c6 running on top of FreeBSD amd64. I forced the
linux_base version to c6 instead of c6_64 in /etc/make.conf and the problems 
went away.
Comment 9 commit-hook freebsd_committer freebsd_triage 2017-01-24 16:14:35 UTC
A commit references this bug:

Author: tijl
Date: Tue Jan 24 16:13:59 UTC 2017
New revision: 312699
URL: https://svnweb.freebsd.org/changeset/base/312699

Log:
  Apply r210555 to 64 bit linux support:

  The interpreter name should no longer be treated as a buffer that can be
  overwritten.

  PR:		216346
  MFC after:	3 days

Changes:
  head/sys/amd64/linux/linux_sysvec.c
Comment 10 Osamu Matsuda 2017-01-25 02:20:31 UTC
(In reply to Tijl Coosemans from comment #7)

Thank you very much.
I'm happy to tell you that it works perfectly with the modification on
/usr/local/Wolfram/Mathematica/10.0/Executables/math as you suggested.

Here is the summary for one of the easiest ways to install Mathematica
10.0 with linux_base-c7.

1) Install linux_base-c7
2) Install mathematica
3) For the following two files, modify the first line from
  #!/bin/sh
to
  #!/compat/linux/bin/sh

  /usr/local/Wolfram/Mathematica/10.0/Executables/mathematica
  /usr/local/Wolfram/Mathematica/10.0/Executables/math

I confirmed this with FreeBSD 10.3-RELEASE-p11,
linux_base-c7-7.3.1611, and Mathematica 10.0.1.0.

But some ad hoc additional modification might be necessary depending on the
cases.  For example, for FreeBSD 10.3-RELEASE-p11, linux_base-c7-7.3.1611, and
Mathematica 10.2.0.0, I had to apply the modification mentioned in 3)
to other files in /usr/local/Wolfram/Mathematica/10.2/Executables.

It is also still uncertain whether we need to modify the symbolic link at
/usr/local/bin/MathematicaScript. which is originally linked to
nonexisting
/usr/local/Wolfram/Mathematica/10.{0,2}/SystemFiles/Kernel/Binaries/Unknown/MathematicaScript,
to either of
/usr/local/Wolfram/Mathematica/10.{0,2}/SystemFiles/Kernel/Binaries/{Linux,Linux-x86-64}/MathematicaScript
Comment 11 Tijl Coosemans freebsd_committer freebsd_triage 2017-01-25 14:30:43 UTC
(In reply to Osamu Matsuda from comment #10)
For FreeBSD head it's no longer needed to change /bin/sh into /compat/linux/bin/sh.  I'll merge that to stable/10 and stable/11 in two days.

To fix /usr/local/bin/MathematicaScript try running the install script using "/compat/linux/bin/sh /path/to/installscript".  It is probably only used with Mathematica shell scripts:
https://www.wolfram.com/mathematica/new-in-8/mathematica-shell-scripts/
https://www.wolfram.com/mathematica/new-in-8/mathematica-shell-scripts/create-mathematica-script-from-your-code.html
Comment 12 commit-hook freebsd_committer freebsd_triage 2017-01-27 10:05:43 UTC
A commit references this bug:

Author: tijl
Date: Fri Jan 27 10:05:17 UTC 2017
New revision: 312873
URL: https://svnweb.freebsd.org/changeset/base/312873

Log:
  MFC r312699:

  Apply r210555 to 64 bit linux support:

  The interpreter name should no longer be treated as a buffer that can be
  overwritten.

  PR:		216346

Changes:
_U  stable/10/
  stable/10/sys/amd64/linux/linux_sysvec.c
Comment 13 commit-hook freebsd_committer freebsd_triage 2017-01-27 10:06:46 UTC
A commit references this bug:

Author: tijl
Date: Fri Jan 27 10:06:20 UTC 2017
New revision: 312874
URL: https://svnweb.freebsd.org/changeset/base/312874

Log:
  MFC r312699:

  Apply r210555 to 64 bit linux support:

  The interpreter name should no longer be treated as a buffer that can be
  overwritten.

  PR:		216346

Changes:
_U  stable/11/
  stable/11/sys/amd64/linux/linux_sysvec.c
Comment 14 Tijl Coosemans freebsd_committer freebsd_triage 2017-02-05 21:49:18 UTC
With the latest round of changes to the Linux ports, c6 should work in the same way as c7:

FreeBSD/i386:
- Mathematica works without changes

FreeBSD/amd64 stable/10, stable/11 and head:
- Mathematica works without changes

FreeBSD/amd64 10.3 and 11.0:
- With DEFAULT_VERSIONS+=linux=c6 in /etc/make.conf (Linux/i386 is used):
  Mathematica works without changes
- Without DEFAULT_VER... i.e. with default c6_64 or c7 (Linux/amd64 is used):
  Due to a kernel bug the first line in each shell script has to be changed
  from "#! /bin/sh" into "#! /compat/linux/bin/sh".
Comment 15 Osamu Matsuda 2017-02-08 03:21:21 UTC
(In reply to Tijl Coosemans from comment #11)
Thank you.

I confirmed that the procedure launched by
  /compat/linux/bin/sh /path/to/installscript
in stable/11 and linux_base-c7 successfully installs everything including the correct link for the mathematica script.