Bug 168974 - [patch] update lang/pypy from 1.8 to 1.9
Summary: [patch] update lang/pypy from 1.8 to 1.9
Status: Closed FIXED
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: Normal Affects Only Me
Assignee: Michael Scheidell
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2012-06-12 12:10 UTC by David Naylor
Modified: 2012-06-30 00:10 UTC (History)
0 users

See Also:


Attachments
file.diff (82.83 KB, patch)
2012-06-12 12:10 UTC, David Naylor
no flags Details | Diff
pypy.diff (83.30 KB, patch)
2012-06-16 15:32 UTC, David Naylor
no flags Details | Diff
pypy.patch (23.65 KB, patch)
2012-06-23 10:51 UTC, Michael Scheidell
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description David Naylor 2012-06-12 12:10:07 UTC
This port update includes:
 - detection of insufficient memory
 - support for CLI (mono) and JVM (java) backends (both backends are currently broken)

Fix: Patch attached with submission follows:
Comment 1 Michael Scheidell freebsd_committer freebsd_triage 2012-06-16 13:32:11 UTC
Responsible Changed
From-To: freebsd-ports-bugs->scheidell

I'll take it.
Comment 2 Michael Scheidell freebsd_committer freebsd_triage 2012-06-16 13:33:25 UTC
State Changed
From-To: open->feedback

Please update your ports tree, and resubmit your patch as an attachment to 
bug-followup@FreeBSD.org with subject line: Re: ports/168974: [patch] update lang/pypy from 1.8 to 1.9 

1) this indicates that some portmgr patch has been added to your port since you last checked: 

-# $FreeBSD: ports/lang/pypy/Makefile,v 1.3 2012/03/31 18:41:26 pav Exp $ 
+# $FreeBSD: ports/lang/pypy/Makefile,v 1.1 2011/12/13 15:37:36 lwhsu Exp $ 
# 


2) you should not touch this: 

-- /usr/ports/lang/pypy/files/use.pypy     2011-12-13 17:37:36.000000000 +0200 
+++ pypy/files/use.pypy     2012-06-12 12:42:38.000000000 +0200 
@@ -1,6 +1,6 @@ 
#!/bin/sh 

-# $FreeBSD: ports/lang/pypy/files/use.pypy,v 1.1 2011/12/13 15:37:36 lwhsu Exp $ 
+# $FreeBSD$ 

3) you have this in makefile: 
test: patch 
.for inst in ${PYPY_INST} 
@${WHICH} ${PYPY_${inst}_NAME} > /dev/null 2>&1 || (${ECHO} "Unable to find ${PYPY_${inst}_NAME}, please install port  
first!"; exit 1) 
(${CD} ${WRKSRC}/lib-python; ${PYPY_${inst}_NAME} ../pypy/test_all.py) 

would it be better to use a BUILD_DEPENDS? 


4) you should use %% vars for these (anything with a 2.7 in it!) 

@dirrm %%PYPYPREFIX%%/lib-python/2.7/xml/sax 
@dirrm %%PYPYPREFIX%%/lib-python/2.7/xml/parsers 
@dirrm %%PYPYPREFIX%%/lib-python/2.7/xml/etree 

(hint, make -V PLIST_SUB), and in Makefile, PLIST_SUB+= 

5) also: is there no other way to do this? we (you and I ) are gong to get called on this by the ports audit committee: 

# Use pypy if it is installed, else use python (to translate) 
.if !defined(PY) 
.if !defined(PYPY) 
.if ${PYPY_PRIMARY} == pypy 
PYPY!=          ${WHICH} ${PYPY_PRIMARY} 2> /dev/null || true 
.else 
PYPY!=          ${WHICH} ${PYPY_PRIMARY} 2> /dev/null || ${WHICH} pypy 2> /dev/null || true 
.endif 
.endif 
.if exists(${PYPY}) 

(this causes a lot of problems with make INDEX, and can usually be handled with an .include "something from other port" file) 

6) please use portlint -abt and fix all you can. (load a new copy of portlint, many changes made to support optionsng) 
this one can ignore:WARN: Makefile: LIB_DEPENDS don't specify the ABI version number .1 in gc.1 unless it is really necessary. 

7) please either submit a completely new patch (via email, attachment,  
to: 
bug-followup@FreeBSD.org 
subject line: 
Re: ports/168974: [patch] update lang/pypy from 1.8 to 1.9 

as an ascii/text attachment. 

if you cannot do this in the next 14 days, then ask that this pr be closed, and open a new one.
Comment 3 David Naylor 2012-06-16 15:32:16 UTC
An updated patch, pending revision based on further feedback.  

P.S. please excuse the TODO list included in the patch.  
Comment 4 Michael Scheidell freebsd_committer freebsd_triage 2012-06-16 16:44:09 UTC
On 6/16/12 10:38 AM, David Naylor wrote:
> Hi,
>
> Thanks for your feedback.  Could you please clarify some things below?
>
> Regards
3), ok.

>> 4) you should use %% vars for these (anything with a 2.7 in it!)
>>
>> @dirrm %%PYPYPREFIX%%/lib-python/2.7/xml/sax
>>   @dirrm %%PYPYPREFIX%%/lib-python/2.7/xml/parsers
>>   @dirrm %%PYPYPREFIX%%/lib-python/2.7/xml/etree
>>
>> (hint, make -V PLIST_SUB), and in Makefile, PLIST_SUB+=
> Just to clarify, the 2.7 comes from the version of python this port supports,
> not which version of python has used to create this port.  Also, python is EOL
> for the 2.x series and the 3.x support from pypy is a long time coming (in
> years) and will likely be a separate port.
>
> With the above said, do you still consider it important that I change this?
>
the way it is, it will only work with python2.7, right? or did I miss 
something?

   %%PYPYPREFIX%%/lib-python/2.7/distutils/spawn.pyc
   %%PYPYPREFIX%%/lib-python/2.7/distutils/sysconfig.py
   %%PYPYPREFIX%%/lib-python/2.7/distutils/sysconfig.pyc

>
> I've submitted a patch dealing with 1, 2 and ~6.  I will follow up with a
> patch dealing with 3, 4 and 5 when you provide some clarity on those points.
5) looks like you dealt with that.

I'll run it in a tinderbox, and see what it does.
(you mentioned that this won't build right on pointyhat in your 
makefile... it MIGHT be the /lib-python/2.7 thing above.
let me see if I can get you better logs.


-- 
Michael Scheidell, CTO
 >*| * SECNAP Network Security Corporation
d: +1.561.948.2259
w: http://people.freebsd.org/~scheidell
Comment 5 Michael Scheidell freebsd_committer freebsd_triage 2012-06-16 17:09:24 UTC
With new patch:

<http://lorie.secnap.net/tb/errors/9-scheidell_AMD64/pypy-1.9.log>
<http://lorie.secnap.net/tb/errors/7-scheidell_I386/pypy-1.9.log>

snip from build logs:

echo>>  /work/a/ports/lang/pypy/work/Makefile
===>   Building for pypy-1.9
/bin/rm -rf build_pypy
/bin/mkdir -p build_pypy
(cd /work/a/ports/lang/pypy/work/pypy-pypy-341e1e3821ff/pypy/translator/goal;  /usr/bin/env TMPDIR=/work/a/ports/lang/pypy/work/build_pypy   translate.py --source  -Ojit  targetpypystandalone.py  )
env: translate.py: No such file or directory
*** Error code 127
Stop in /work/a/ports/lang/pypy/work.


*** Error code 1

basically, this says to use this port if this port is installed? else 
use python? and python takes 2+ hours to translate ?
you need a way to boot strap this then.
(this looks like it sets USE_PYTHON_BUILD,  so, do you need to call 
python to run this translate during a build?)

  Use pypy if it is installed, else use python (to translate)
.if !defined(PY)
.if !defined(PYPY)
.if ${PYPY_PRIMARY} == pypy
PYPY!=          ${WHICH} ${PYPY_PRIMARY} 2> /dev/null || true
.else
PYPY!=          ${WHICH} ${PYPY_PRIMARY} 2> /dev/null || ${WHICH} pypy 
2> /dev/null || true
.endif
.endif
.if exists(${PYPY})
PY=             ${PYPY}
.else
USE_PYTHON_BUILD=       2.5+
PY=             ${PYTHON_CMD}
.endif
.endif


-- 
Michael Scheidell, CTO
 >*| * SECNAP Network Security Corporation
d: +1.561.948.2259
w: http://people.freebsd.org/~scheidell
Comment 6 David Naylor 2012-06-22 10:32:28 UTC
Apologies for the delay.  It appears my email address was dropped from the 
replies and I didn't check the bug...

I am working on an update that I think will result in a cleaner port without 
the need for a specific $PREFIX/pypy-1.9 directory.  

In reply to the issues raised:

On 2012/06/16 11:44 AM Michael Scheidell wrote:
>> With the above said, do you still consider it important that I change this?
>>
> the way it is, it will only work with python2.7, right? or did I miss 
> something?


Pypy doesn't need python2.7 (or any python) once installed.  Pypy is an 
implementation of the python 2.7 language and libraries.  

Pypy, however, does require a valid python binary to translate (build) itself.  
Since pypy itself is such a binary it can be used to translate itself, 
otherwise it can use cpython (or jython, or ironpython, in theory).  

> %%PYPYPREFIX%%/lib-python/2.7/distutils/spawn.pyc
> %%PYPYPREFIX%%/lib-python/2.7/distutils/sysconfig.py
> %%PYPYPREFIX%%/lib-python/2.7/distutils/sysconfig.pyc


I would ideally like to move %%PYPYPREFIX%%/lib-python/2.7 to somewhere under 
$PREFIX/lib.  Following the convention of lang/python it would be 
$PREFIX/lib/pypy1.9, except pypy has to library folders (lib-python and 
lib_pypy) and I currently don't know how to change the search path.  

On 2012/06/16 12:09 AM Michael Scheidell wrote:
>With new patch:
>
> <http://lorie.secnap.net/tb/errors/9-scheidell_AMD64/pypy-1.9.log>
> <http://lorie.secnap.net/tb/errors/7-scheidell_I386/pypy-1.9.log>
>
> snip from build logs:
>
> echo>> /work/a/ports/lang/pypy/work/Makefile
> ===> Building for pypy-1.9
> /bin/rm -rf build_pypy
> /bin/mkdir -p build_pypy
> (cd /work/a/ports/lang/pypy/work/pypy- 
> pypy-341e1e3821ff/pypy/translator/goal; /usr/bin/env  
> TMPDIR=/work/a/ports/lang/pypy/work/build_pypy translate.py --source -Ojit 
> targetpypystandalone.py )
> env: translate.py: No such file or directory
> *** Error code 127
> Stop in /work/a/ports/lang/pypy/work.
>
> *** Error code 1


I wondered why I had used ".include <bsd.port.options.mk>".  This is why.  I 
will fix this...

> basically, this says to use this port if this port is installed? else 
> use python? and python takes 2+ hours to translate ?


~yes.  Pypy translates faster than python, but it also uses more memory.  

> you need a way to boot strap this then.
> (this looks like it sets USE_PYTHON_BUILD, so, do you need to call 
> python to run this translate during a build?)


Yes, either pypy or python can be used (either are interchangeable regarding 
the build).  

> Use pypy if it is installed, else use python (to translate)
> .if !defined(PY)
> .if !defined(PYPY)
> .if ${PYPY_PRIMARY} == pypy
> PYPY!= ${WHICH} ${PYPY_PRIMARY} 2> /dev/null || true
> .else
> PYPY!= ${WHICH} ${PYPY_PRIMARY} 2> /dev/null || ${WHICH} pypy 
> 2> /dev/null || true
> .endif
> .endif
> .if exists(${PYPY})
> PY= ${PYPY}
> .else
> USE_PYTHON_BUILD= 2.5+
> PY= ${PYTHON_CMD}
> .endif
> .endif
Comment 7 Michael Scheidell freebsd_committer freebsd_triage 2012-06-23 10:51:08 UTC
Uncompressed patch attached for archives/logs:

-- 
Michael Scheidell, CTO
SECNAP Network Security Corporation
http://people.freebsd.org/~scheidell

______________________________________________________________________
This email has been scanned and certified safe by SpammerTrap(r). 
For Information please see http://www.spammertrap.com/
______________________________________________________________________  
  
Comment 8 Michael Scheidell freebsd_committer freebsd_triage 2012-06-23 10:53:27 UTC
Responsible Changed
From-To: scheidell->freebsd-ports-bugs

Back to the pool. 
you need to attach the unpacked, uncompressed, unmangled patch
Comment 9 Michael Scheidell freebsd_committer freebsd_triage 2012-06-29 18:23:46 UTC
Responsible Changed
From-To: freebsd-ports-bugs->scheidell

I'll take it.
Comment 10 dfilter service freebsd_committer freebsd_triage 2012-06-30 00:06:17 UTC
scheidell    2012-06-29 23:06:06 UTC

  FreeBSD ports repository

  Modified files:
    lang/pypy            Makefile distinfo pkg-plist 
    lang/pypy/files      bsd.pypy.inst.mk 
                         patch-pypy__rpython__tool__rffi_platform.py 
                         use.pypy 
  Added files:
    lang/pypy            TODO 
    lang/pypy/files      
                         patch-lib__pypy1.9__lib_pypy__ctypes_config_cache__rebuild.py 
                         patch-lib__pypy1.9__site.py 
                         patch-py___path__local.py 
                         patch-pypy__module__sys__state.py 
                         patch-pypy__tool__lib_pypy.py 
                         patch-pypy__translator__cli__sdk.py 
  Removed files:
    lang/pypy/files      patch-CVE-2012-0845 
                         patch-pypy__ctypes_configure__cbuild.py 
  Log:
  - Update to 1.9 [1]
  - Detection of insufficient memory [1]
  - Change %%  SUB vaes from fixed at python 27 to use any installed version of python [1]
  - Fix syntax of non system include "" vs <> [2]
  
  PR:             ports/168974 [1]
  Submitted by:   David Naylor <naylor.b.david@gmail.com> (maintainer) [1]
  Reviewed by:    scheidel@ (me) [2]
  
  Revision  Changes      Path
  1.4       +94 -43      ports/lang/pypy/Makefile
  1.1       +8 -0        ports/lang/pypy/TODO (new)
  1.3       +2 -2        ports/lang/pypy/distinfo
  1.3       +5 -0        ports/lang/pypy/files/bsd.pypy.inst.mk
  1.2       +0 -14       ports/lang/pypy/files/patch-CVE-2012-0845 (dead)
  1.1       +11 -0       ports/lang/pypy/files/patch-lib__pypy1.9__lib_pypy__ctypes_config_cache__rebuild.py (new)
  1.1       +17 -0       ports/lang/pypy/files/patch-lib__pypy1.9__site.py (new)
  1.1       +12 -0       ports/lang/pypy/files/patch-py___path__local.py (new)
  1.2       +0 -57       ports/lang/pypy/files/patch-pypy__ctypes_configure__cbuild.py (dead)
  1.1       +21 -0       ports/lang/pypy/files/patch-pypy__module__sys__state.py (new)
  1.2       +1 -1        ports/lang/pypy/files/patch-pypy__rpython__tool__rffi_platform.py
  1.1       +18 -0       ports/lang/pypy/files/patch-pypy__tool__lib_pypy.py (new)
  1.1       +11 -0       ports/lang/pypy/files/patch-pypy__translator__cli__sdk.py (new)
  1.2       +4 -4        ports/lang/pypy/files/use.pypy
  1.3       +3848 -4907  ports/lang/pypy/pkg-plist
_______________________________________________
cvs-all@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/cvs-all
To unsubscribe, send any mail to "cvs-all-unsubscribe@freebsd.org"
Comment 11 Michael Scheidell freebsd_committer freebsd_triage 2012-06-30 00:06:24 UTC
State Changed
From-To: feedback->closed

Committed, with minor changes. Thanks!