Bug 201523 - lang/python27 on raspberry B, _ctypes libffi sysv.S 'invalid instruction'
Summary: lang/python27 on raspberry B, _ctypes libffi sysv.S 'invalid instruction'
Status: Closed Feedback Timeout
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: arm Any
: --- Affects Only Me
Assignee: Walter Schwarzenfeld
URL: https://github.com/atgreen/libffi/iss...
Keywords: needs-qa, patch
Depends on:
Blocks:
 
Reported: 2015-07-13 12:09 UTC by ArnoB
Modified: 2019-08-27 15:50 UTC (History)
9 users (show)

See Also:
koobs: maintainer-feedback+
koobs: merge-quarterly?


Attachments
a proper diff -u (422 bytes, patch)
2015-07-13 13:04 UTC, ArnoB
no flags Details | Diff
make (495.35 KB, text/plain)
2015-07-16 10:35 UTC, ArnoB
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description ArnoB 2015-07-13 12:09:07 UTC
compiling lang/python27 fails to compile _ctypes.so due to a non existent assembly instruction:
...
/usr/ports/lang/python27/work/Python-2.7.10/Modules/_ctypes/libffi/src/arm/sysv.S:399:2: error: invalid instruction
 stmeqia r2, {r0, r1}
 ^
...


i found a similar report online:
http://lists.freedesktop.org/archives/gstreamer-commits/2013-September/074531.html

so i replaced line 399:
    stmeqia r2, {r0, r1}
with:
    stmiaeq r2, {r0, r1}

and then it compiles fine.


# diff sysv.S sysv.S.new 
399c399
< 	stmeqia	r2, {r0, r1}
---
> 	stmiaeq	r2, {r0, r1}
Comment 1 ArnoB 2015-07-13 13:04:36 UTC
Created attachment 158689 [details]
a proper diff -u
Comment 2 Kubilay Kocak freebsd_committer freebsd_triage 2015-07-14 03:50:44 UTC
(In reply to ArnoB from comment #0)

 * What version of FreeBSD? (uname -a output)
 * Are you using the lang/python27 port or package?
 * If using the lang/python27 port, is it built with or without LIBFFI option?
 * If built without LIBFFI option, please try to reproduce after rebuilding with the LIBFFI option enabled

Thanks
Comment 3 ArnoB 2015-07-14 13:30:27 UTC
(In reply to Kubilay Kocak from comment #2)

 * What version of FreeBSD? (uname -a output)

FreeBSD rpi-b 11.0-CURRENT FreeBSD 11.0-CURRENT #0 r284969: Wed Jul  1 06:34:08 UTC 2015     root@releng2.nyi.freebsd.org:/usr/obj/arm.armv6/usr/src/sys/RPI-B  arm


 * Are you using the lang/python27 port or package?

I'm installing from ports: lang/python27
(It was actually initated by the installation of subversion)


 * If using the lang/python27 port, is it built with or without LIBFFI option?

I didn't get an option to build it without. Only to use python's bundled version or the one from ports.
make config gave me this:
"LIBFFI    Use libffi from ports instead of bundled version"
I have this disabled.


 * If built without LIBFFI option, please try to reproduce after rebuilding with the LIBFFI option enabled
I tried enabling it, but that didn't make a difference. I got the exact same error.
Comment 4 Kubilay Kocak freebsd_committer freebsd_triage 2015-07-15 10:05:29 UTC
cc sean, original reporter of upstream issue
Comment 5 Sean Bruno freebsd_committer freebsd_triage 2015-07-15 21:40:17 UTC
From my recollection, either we dropped suport for the built in libffi .  I don't see this issue with building python for arm any longer.

http://chips.ysv.freebsd.org/data/11armv6-default/2015-07-11_19h23m27s/logs/python27-2.7.10.log
Comment 6 Kubilay Kocak freebsd_committer freebsd_triage 2015-07-16 02:47:56 UTC
(In reply to Sean Bruno from comment #5)

The LIBFFI option was added and was/is enabled by default.

The option switches Python to use the the ports version of libffi (devel/libffi).

The option can be disabled, which results in Python using its bundled version. 

If the issue is not reproducible for you with this option ENABLED, then either:

 * A change fixing the issue has already been committed in libffi upstream and released (and the bug should not be open), OR
 * A change was made elsewhere outside of Python/libffi (FreeBSD?) that fixes it.

If the issue is not reproducible for you with this option DISABLED, then either:

 * A fix was committed to Python upstream, OR
 * A change was made elsewhere outside of Python/libffi (FreeBSD?) that fixes it.

Either way, we need to isolate why this works for you but doesn't for the original reporter
Comment 7 Kubilay Kocak freebsd_committer freebsd_triage 2015-07-16 02:48:57 UTC
Arno, can you include (as an attachment) the ports *full* build log please
Comment 8 ArnoB 2015-07-16 10:35:19 UTC
Created attachment 158837 [details]
make
Comment 9 ArnoB 2015-07-16 10:37:07 UTC
(In reply to Kubilay Kocak from comment #7)

sure: python27.fail.txt attached...


it's the log from 'make'.
near the end it shows the error failing to build ctypes.

'make install' afterwards failes complaining about the missing _ctypes.so
Comment 10 Chris Rees freebsd_committer freebsd_triage 2016-10-28 20:15:50 UTC
Still an issue :)

https://www.bayofrum.net/pkg/110armv6-system/2016-10-25_21h19m52s/logs/errors/python27-2.7.12.log

How about we just remove the ffi option?

Chris
Comment 11 Kyle Evans freebsd_committer freebsd_triage 2016-10-28 20:54:21 UTC
As another data point, I build (and rebuild) lang/python27 with Poudriere for 11.0-RELEASE ARMv6 (as a part of building www/py-octoprint) frequently and do not have this problem.
Comment 12 Chris Rees freebsd_committer freebsd_triage 2016-10-28 20:58:19 UTC
Cheers Kyle- you are evidently compiling with LIBFFI enabled, yes?
Comment 13 Kyle Evans freebsd_committer freebsd_triage 2016-10-28 21:32:35 UTC
(In reply to Chris Rees from comment #12)

Ugh, I thought for some reason I had disabled this. =( Nope, it was enabled, and definitely fails once disabled. Apologies for the extra noise.
Comment 14 Kubilay Kocak freebsd_committer freebsd_triage 2016-10-29 07:21:34 UTC
Removing the option for LIBFFI is a workaround. It's preferable the root cause is addressed. Since no-one addressed the reproduction/isolation questions in comment 6 (the reason this is still an open issue), we can proceed as follows:

a) Add a BROKEN condition for LIBFFI=off & ARCH=foo||bar as we do for LIBFFI/i386 currently, OR

b) Patch lang/python* portslocally referencing the upstream issues, eventually resulting in the patch not being required once 3.3.x is released and Python no longer bundles libffi (which they intend to do [1])

Option (b) is preferable (root cause fix), but in order to do that properly, we need an answer to the following question:

Q) Is this a problem only on lang/python2.7 or on every version up to and including lang/python35 (and the Python 3.6 port I intend to commit soon)

I'm happy to commit (a) in the meantime until we have that answer, but the following is needed:

a) Confirmation is needed from those who are having the issue that this is a acceptable response/resolution.

b) A list of all python port versions and all architectures which this has been reported to be, would be, or are affected.

[1] http://bugs.python.org/issue28491
Comment 15 Kubilay Kocak freebsd_committer freebsd_triage 2016-10-29 07:23:07 UTC
Don't In Progress without a (real) Assignee otherwise it will fall through the cracks.

I'm happy to assign and resolve this issue based on feedback requested in comment 14
Comment 16 Chris Rees freebsd_committer freebsd_triage 2016-10-29 07:31:04 UTC
I must admit I still think we should just remove the LIBFFI option.

The best solution in my opinion is to have it as an amd64 option only, especially as it's a transient problem, and upstream clearly doesn't want us using it anyway.

Chris
Comment 17 Kubilay Kocak freebsd_committer freebsd_triage 2016-10-29 07:45:19 UTC
(In reply to Chris Rees from comment #16)

Removing the option when that becomes the default (3.7) is perfectly reasonable. Removing it prior is over-inclusive. A BROKEN for the option/arch combination (option (a) in comment 14) addresses the issue as reported/isolated.

If the issue affects more than just arm, or a case can be made that the issue is a result of a single-ARCH-specific instruction being used and there is no practical workaround or fix, sure, limiting the LIBFFI option to that specific ARCH then becomes an understandable request.

In the meantime the libffi option (both in the port and upstreap) as well as the ability to switch between the default (bundled) and a system version via configure is entirely supported.

All that is required is list of ARCHS and Python ports for which this is an issue, as we have a fix for the problem in attachment 158689 [details]

What won't happen is partial resolution, as that creates more work for people down the track. In particular, given the issue is not reproducible in the OPTIONS_DEFAULT case, the urgency and warrant to workaround fix this is greatly diminished.
Comment 18 Kubilay Kocak freebsd_committer freebsd_triage 2016-10-29 07:50:00 UTC
I note that again, option (b) is preferable as it solves the root cause, properly and completely. I just need to know whether it's an issue on all python ports, or just python27, but I do not have the hardware in question. I'm assuming it is all, but require that confirmation.
Comment 19 Christoph Sold 2017-04-07 13:38:58 UTC
Building lang/python36 with base libffi works on Raspberry Pi (arm), python27 still doesn't.
$ uname -a
FreeBSD papas-rpi2 11.0-STABLE FreeBSD 11.0-STABLE #0 r313108: Fri Feb  3 08:15:17 UTC 2017     root@releng2.nyi.freebsd.org:/usr/obj/arm.armv6/usr/src/sys/RPI2  arm
Comment 20 Eitan Adler freebsd_committer freebsd_triage 2018-05-23 10:27:25 UTC
batch change of PRs untouched in 2018 marked "in progress" back to open.
Comment 21 Walter Schwarzenfeld freebsd_triage 2019-02-15 23:32:38 UTC
Is this still relevant?
Comment 22 Walter Schwarzenfeld freebsd_triage 2019-08-27 15:50:32 UTC
Close with feedback timeout. Please reopen if there still problems.