Bug 202159 - lang/pypy: Broken on ARM64 and ARMv6 (unknown machine name %s)
Summary: lang/pypy: Broken on ARM64 and ARMv6 (unknown machine name %s)
Status: Closed FIXED
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: arm64 Any
: --- Affects Some People
Assignee: David Naylor
Keywords: needs-qa
Depends on:
Blocks: 201763
  Show dependency treegraph
Reported: 2015-08-07 18:02 UTC by Sean Bruno
Modified: 2015-08-13 18:05 UTC (History)
5 users (show)

See Also:
bugzilla: maintainer-feedback? (dbn)

arm fix (567 bytes, patch)
2015-08-09 21:08 UTC, Mikael Urankar
no flags Details | Diff
[WIP] Update to lang/pypy (129 bytes, patch)
2015-08-11 19:24 UTC, David Naylor
no flags Details | Diff
[WIP] Update to lang/pypy (7.18 KB, patch)
2015-08-12 04:37 UTC, David Naylor
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Sean Bruno freebsd_committer 2015-08-07 18:02:53 UTC
[platform:execute] /nxb-bin/usr/bin/cc -shared /wrkdirs/usr/ports/lang/pypy/work/build/usession-release-2.6.0-0/rpython/translator/c/src/thread.o -L/usr/local/lib -L/usr/local/lib -pthread -Wl,--export-dynamic -o /wrkdirs/usr/ports/lang/pypy/work/build/usession-release-2.6.0-0/shared_cache/externmod_0.so
[platform:execute] /nxb-bin/usr/bin/cc -c -O3 -pthread -fomit-frame-pointer -O2 -pipe -fno-strict-aliasing -I/usr/local/include -I/usr/local/include -I/usr/local/include /wrkdirs/usr/ports/lang/pypy/work/build/usession-release-2.6.0-0/platcheck_48.c -o /wrkdirs/usr/ports/lang/pypy/work/build/usession-release-2.6.0-0/platcheck_48.o
[platform:execute] /nxb-bin/usr/bin/cc /wrkdirs/usr/ports/lang/pypy/work/build/usession-release-2.6.0-0/platcheck_48.o -L/usr/local/lib -L/usr/local/lib -L/usr/local/lib -pthread -Wl,--export-dynamic -lexpat -o /wrkdirs/usr/ports/lang/pypy/work/build/usession-release-2.6.0-0/platcheck_48
[platform:execute] /nxb-bin/usr/bin/cc -c -O3 -pthread -fomit-frame-pointer -O2 -pipe -fno-strict-aliasing -I/usr/local/include -I/usr/local/include -I/usr/local/include /wrkdirs/usr/ports/lang/pypy/work/build/usession-release-2.6.0-0/platcheck_49.c -o /wrkdirs/usr/ports/lang/pypy/work/build/usession-release-2.6.0-0/platcheck_49.o
[platform:execute] /nxb-bin/usr/bin/cc /wrkdirs/usr/ports/lang/pypy/work/build/usession-release-2.6.0-0/platcheck_49.o -L/usr/local/lib -L/usr/local/lib -L/usr/local/lib -pthread -Wl,--export-dynamic -lexpat -o /wrkdirs/usr/ports/lang/pypy/work/build/usession-release-2.6.0-0/platcheck_49
[translation:info] Error:
[translation:info]    File "/wrkdirs/usr/ports/lang/pypy/work/pypy-pypy-295ee98b6928/rpython/translator/goal/translate.py", line 284, in main
[translation:info]     default_goal='compile')
[translation:info]    File "/wrkdirs/usr/ports/lang/pypy/work/pypy-pypy-295ee98b6928/rpython/translator/driver.py", line 562, in from_targetspec
[translation:info]     spec = target(driver, args)
[translation:info]    File "targetpypystandalone.py", line 295, in target
[translation:info]     return self.get_entry_point(config)
[translation:info]    File "targetpypystandalone.py", line 307, in get_entry_point
[translation:info]     space = make_objspace(config)
[translation:info]    File "/wrkdirs/usr/ports/lang/pypy/work/pypy-pypy-295ee98b6928/pypy/tool/option.py", line 35, in make_objspace
[translation:info]     return Space(config)
[translation:info]    File "/wrkdirs/usr/ports/lang/pypy/work/pypy-pypy-295ee98b6928/pypy/interpreter/baseobjspace.py", line 395, in __init__
[translation:info]     self.initialize()
[translation:info]    File "/wrkdirs/usr/ports/lang/pypy/work/pypy-pypy-295ee98b6928/pypy/objspace/std/objspace.py", line 105, in initialize
[translation:info]     self.setup_builtin_modules()
[translation:info]    File "/wrkdirs/usr/ports/lang/pypy/work/pypy-pypy-295ee98b6928/pypy/interpreter/baseobjspace.py", line 633, in setup_builtin_modules
[translation:info]     mod.setup_after_space_initialization()
[translation:info]    File "/wrkdirs/usr/ports/lang/pypy/work/pypy-pypy-295ee98b6928/pypy/module/__pypy__/__init__.py", line 116, in setup_after_space_initialization
[translation:info]     model = detect_cpu.autodetect()
[translation:info]    File "/wrkdirs/usr/ports/lang/pypy/work/pypy-pypy-295ee98b6928/rpython/jit/backend/detect_cpu.py", line 98, in autodetect
[translation:info]     return detect_model_from_host_platform()
[translation:info]    File "/wrkdirs/usr/ports/lang/pypy/work/pypy-pypy-295ee98b6928/rpython/jit/backend/detect_cpu.py", line 69, in detect_model_from_host_platform
[translation:info]     raise ProcessorAutodetectError, "unknown machine name %s" % mach
[translation:ERROR] ProcessorAutodetectError: unknown machine name arm64
[translation] start debugger...
> /wrkdirs/usr/ports/lang/pypy/work/pypy-pypy-295ee98b6928/rpython/jit/backend/detect_cpu.py(69)detect_model_from_host_platform()
-> raise ProcessorAutodetectError, "unknown machine name %s" % mach
Traceback (most recent call last):
  File "../../rpython/bin/rpython", line 20, in <module>
  File "/wrkdirs/usr/ports/lang/pypy/work/pypy-pypy-295ee98b6928/rpython/translator/goal/translate.py", line 325, in main
  File "/wrkdirs/usr/ports/lang/pypy/work/pypy-pypy-295ee98b6928/rpython/translator/goal/translate.py", line 278, in debug
  File "/wrkdirs/usr/ports/lang/pypy/work/pypy-pypy-295ee98b6928/rpython/translator/tool/pdbplus.py", line 441, in start
  File "/wrkdirs/usr/ports/lang/pypy/work/pypy-pypy-295ee98b6928/rpython/translator/tool/pdbplus.py", line 25, in post_mortem
    self.interaction(t.tb_frame, t)        
  File "/usr/local/lib/python2.7/pdb.py", line 210, in interaction
  File "/usr/local/lib/python2.7/cmd.py", line 109, in cmdloop
  File "/wrkdirs/usr/ports/lang/pypy/work/pypy-pypy-295ee98b6928/rpython/translator/tool/pdbplus.py", line 29, in preloop
    raise NoTTY("Cannot start the debugger when stdout is captured.")
rpython.translator.tool.pdbplus.NoTTY: Cannot start the debugger when stdout is captured.
*** Error code 1

make: stopped in /usr/ports/lang/pypy
=======================<phase: Interactive    >============================
[00:20:35] ====>> Installing packages
[00:20:35] ====>> Installing run-depends for lang/pypy
[00:20:36] ====>> Installing lang/pypy
pkg-static: /tmp/pkgs/pypy-2.6.0_2.txz: No such file or directory
pkg-static: Was 'pkg install /tmp/pkgs/pypy-2.6.0_2.txz' meant?

Failed to install the following 1 package(s): /tmp/pkgs/pypy-2.6.0_2.txz
*** Error code 70

make: stopped in /usr/ports/lang/pypy
[00:20:37] ====>> Warning: Failed to install lang/pypy
[00:20:37] ====>> Installing local Pkg repository to /usr/local/etc/pkg/repos
[00:20:37] ====>> Leaving jail 11aarch64-default-n running, mounted at /usr/local/poudriere/data/.m/11aarch64-default/ref for interactive run testing
[00:20:37] ====>> To enter jail: jexec 11aarch64-default-n env -i TERM=$TERM /usr/bin/login -fp root
[00:20:37] ====>> To stop jail: poudriere jail -k -j 11aarch64
Comment 1 Sean Bruno freebsd_committer 2015-08-07 18:19:31 UTC
I see the same failure on armv6
Comment 2 Kubilay Kocak freebsd_committer freebsd_triage 2015-08-08 02:43:04 UTC
In the meantime this should be set to broken on arm64 (or only for archs: x, y, z) while it's investigated / resolved.
Comment 3 Sean Bruno freebsd_committer 2015-08-09 18:50:17 UTC
(In reply to Kubilay Kocak from comment #2)

Note that for ARMv6, this seems to be a regression in the last update.

good build:

bad build:
Comment 4 Mikael Urankar freebsd_committer 2015-08-09 21:08:12 UTC
Created attachment 159710 [details]
arm fix

I'm able to build it on armv6 with the attached patch
Comment 5 Andrew Turner freebsd_committer 2015-08-09 21:45:18 UTC
The arm64 part of the patch looks wrong, it's a different instruction set than 32-bit arm so I would expect it to fail.
Comment 6 David Naylor freebsd_committer 2015-08-10 19:09:14 UTC
@Andrew: It appears PyPy only supports a single "arm" mode, thus the change will be consistent with how PyPy handles ARM.  I'll try and commit this fix tomorrow evening (I also have other changes in the pipeline).  

Would anyone be able to build test the patch from Mikael?
Comment 7 Sean Bruno freebsd_committer 2015-08-11 18:14:35 UTC
(In reply to David Naylor from comment #6)
Running builds now, emulation of the lang/pypy build take a looooong time.


Comment 8 Sean Bruno freebsd_committer 2015-08-11 18:44:29 UTC
(In reply to Sean Bruno from comment #7)
The AARCH64 build fails.  Let's remove that from the patch and add a BROKEN entry into the Makefile for AARCH64 so that we don't waste time building it.
Comment 9 David Naylor freebsd_committer 2015-08-11 18:59:02 UTC
(In reply to Sean Bruno from comment #8)

The root cause of the error is that the __arm__ C "macro" isn't defined.  It appears PyPy does not have an arm64 JIT compiler.  

I can suggest two approaches:
 a) Make the port ONLY_FOR_ARCHS= i386 amd64 arm (JIT only available on those archs) - as you suggest, or
 b) Disable the JIT on non-supported architectures (although the JIT is the main reason an end user would want PyPy).  

I agree with your assessment for (a) on based on POLA.
Comment 10 David Naylor freebsd_committer 2015-08-11 19:24:34 UTC
Created attachment 159776 [details]
[WIP] Update to lang/pypy

See attached for my proposed update to PyPy.  I still need to update the pkg-plist (and verify the cffi ports have all the required dependencies).  But, as you say, translation takes a very, very long time...
Comment 11 Sean Bruno freebsd_committer 2015-08-11 23:14:18 UTC
(In reply to David Naylor from comment #10)
The patch attached is just an svn status?
Comment 12 David Naylor freebsd_committer 2015-08-12 04:37:09 UTC
Created attachment 159796 [details]
[WIP] Update to lang/pypy

Comment 13 Sean Bruno freebsd_committer 2015-08-12 15:06:34 UTC
(In reply to David Naylor from comment #12)
+ONLY_FOR_ARCHS=	i386 amd64 arm
+ONLY_FOR_ARCHS_REASON=	PyPy JIT only supported on these architectures

^^ needs to be "armv6" here.  ARCH in ports translates to TARGET_ARCH in src.

-# untested (and do not have jit support).
-.if ${ARCH} == "i386"
-PYPY_ARCH=	x86_32
+.if ${ARCH} == "i386" || ${ARCH} == "arm"

^^ same here, needs to be armv6
Comment 14 commit-hook freebsd_committer 2015-08-12 18:14:20 UTC
A commit references this bug:

Author: dbn
Date: Wed Aug 12 18:13:25 UTC 2015
New revision: 394025
URL: https://svnweb.freebsd.org/changeset/ports/394025

  lang/pypy: support cffi, fix and simplify.

   * Mark as only for i386, amd64 and armv6 (JIT only supported on those archs)
   * Move PyPy dir to pypy26 (for consistency with llvm, et al)
   * Drop PYPY_ARCH (unused)
   * Use shell to detect ${BUILD_WRKSRC}/Makefile [1]
   * Enable CFFI for components that do not have external dependencies
   * Update regression-test to align with upstreams procedures
   * Complete changes for detecting cpu count via sysctl (upstreamed)
   * Properly detect arm (upstreamed) [2]
   * Update pkg-plist generation and regenerate

  [1] When make evaluates the dependency for pre-build the variable
  ${BUILD_WRKSRC} is not fully defined (i.e. ${WRKSRC}) so it does not get
  properly expanded causing make to never find the correct Makefile.

  PR:		202159 [2]
  Reviewed by:	sbruno (prior version)

Comment 15 David Naylor freebsd_committer 2015-08-13 18:05:14 UTC
My build for armv6 also failed due to a lack of memory (in the virtual environment: poudriere + qemu-user-static), however, all indications are that it will compile and run.