Bug 133081 - [exp-run] [PATCH] Fix PYEASYINSTALL_ARCHDEP=yes packages
Summary: [exp-run] [PATCH] Fix PYEASYINSTALL_ARCHDEP=yes packages
Status: Closed FIXED
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: Normal Affects Only Me
Assignee: Chris Rees
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2009-03-26 02:30 UTC by Stef Walter
Modified: 2013-06-29 12:50 UTC (History)
0 users

See Also:


Attachments
file.txt (6.09 KB, text/plain)
2009-03-26 02:30 UTC, Stef Walter
no flags Details
bsd-python-osrel.diff (884 bytes, patch)
2009-03-26 04:05 UTC, Stef Walter
no flags Details | Diff
patch-python-distutils-osrel.diff (519 bytes, patch)
2009-03-26 04:05 UTC, Stef Walter
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Stef Walter 2009-03-26 02:30:03 UTC
The PYEASYINSTALL_ARCHDEP=YES port option makes packages that can only be installed on the same specific patch level of FreeBSD.

For example if you build net/py-ldap2 as a package on FreeBSD 7.1-RELEASE-p3 then it won't install on a FreeBSD 7.1-RELEASE-p4 system.

The following ports are affected:

audio/py-tagpy
databases/py-MySQLdb-devel
databases/py-pg_pqueue
devel/py-jsonlib
devel/py-pylib
devel/py-lxml
devel/py-cheetah
devel/py-fusefs
devel/py-ruledispatch
devel/py-Jinja
devel/py-ll-core
devel/py-rbtree
devel/py-durus
devel/py-rtree
devel/py-Jinja2
devel/py-traits
devel/py-protocols-devel
devel/py-xattr
devel/py-icu
devel/py-simplejson
devel/py-zopeInterface
devel/py-mwlib.ext
devel/py-mwlib
math/py-igraph
net/py-ldap2
net/py-zopeproxy
security/py-m2crypto
security/py-pycryptopp
security/py-posix1e
textproc/py-rdflib
textproc/py-genshi
www/py-amf
www/xist

Fix: Attached are two patches that help fix this problem. 

patch-python-distutils-osrel.diff 

 * place in /usr/ports/lang/python25/files 
 * Should eventually go upstream, but should be patched in python port.
 * Haven't tested with other pythons but patch will be similar.

bsd-python-osrel.diff

 * patch for Mk/bsd.python.mk 

The result of these two patches is that PYEASYINSTALL_ARCHDEP packages built on any 7.x system will install on any 7.x system as expected.


Patch attached with submission follows:
How-To-Repeat: Build any of the above ports.
Comment 1 Stef Walter 2009-03-26 04:05:26 UTC
Let's try attaching these as files, and see if it's more readable.
Comment 2 Pav Lucistnik freebsd_committer 2009-03-26 10:02:44 UTC
Responsible Changed
From-To: freebsd-ports-bugs->freebsd-python

Assign to freebsd-python
Comment 3 Victor Balada Diaz 2012-11-16 20:42:37 UTC
I've just been beaten by this issue while installing sysutils/salt
from pkgng. Salt depends on py-m2crypto, which use PYEASYINSTALL_ARCHDEP.

As pkgng makes using binary packages much easier, this problem will start
affecting more users as new releases start with it by default.

Is there any update on fixing this issue?

Regards.
-- 
La prueba más fehaciente de que existe vida inteligente en otros
planetas, es que no han intentado contactar con nosotros.
Comment 4 Stef Walter 2012-11-26 13:59:39 UTC
On 11/16/2012 09:42 PM, Victor Balada Diaz wrote:
> I've just been beaten by this issue while installing sysutils/salt
> from pkgng. Salt depends on py-m2crypto, which use PYEASYINSTALL_ARCHDEP.
> 
> As pkgng makes using binary packages much easier, this problem will start
> affecting more users as new releases start with it by default.
> 
> Is there any update on fixing this issue?

I got a lot of push back from upstream folks talking about how using
distutils isn't the right way forward and so on and so forth. So I
backed away and left this to be handled by someone more intimately involved.

In many of my previous clients large deployments they've been using
these patches for about 5 years now. Without them the way the
architecture dependent python packages work is just broken.

Cheers,

Stef
Comment 5 Chris Rees 2013-05-30 17:46:01 UTC
The solution to this appears pretty easy on the surface.

Taking Stef's first suggestion on stripping all but the major version,
we can also set the envar _PYTHON_HOST_PLATFORM to make distutils use
that instead.

Do we think this needs an exp-run?  I've tested it on a couple of
ports, and it appears to work well, going from:

cracklib-2.8.19-py2.7-freebsd-9.1-STABLE-amd64.egg/

to

cracklib-2.8.19-py2.7-freebsd-9-amd64.egg/

Chris

http://www.bayofrum.net/~crees/patches/fix-distutils-arch.diff
Comment 6 Chris Rees freebsd_committer 2013-06-04 19:19:41 UTC
Responsible Changed
From-To: freebsd-python->portmgr
Comment 7 Chris Rees freebsd_committer 2013-06-04 20:30:14 UTC
Responsible Changed
From-To: portmgr->crees

Actually, it's only 73 ports that even use PYEASYINSTALL_ARCHDEP; I'll 
put them in Tinderbox myself.  Sorry to bother you all.
Comment 8 Chris Rees 2013-06-06 18:10:41 UTC
Hi all again,

After much grinding of teeth, I have discovered that easy_install
completely ignores the envar _PYTHON_HOST_PLATFORM.

I also discovered that the only thing it really did was add the egg to
%%PYTHON_SITELIBDIR%%/easy-install.pth, so I wrote some sed and ed
magic to replace it.

This patch now fixes the problem in
http://www.freebsd.org/cgi/query-pr.cgi?pr=ports/133081.

I've tested it and it works, and all the affected ports pass tinderbox.

Any comments before I commit?

http://www.bayofrum.net/~crees/patches/python-filthy2.diff

Chris
Comment 9 dfilter service freebsd_committer 2013-06-29 12:43:08 UTC
Author: crees
Date: Sat Jun 29 11:43:01 2013
New Revision: 322046
URL: http://svnweb.freebsd.org/changeset/ports/322046

Log:
  When python ports are installed that depend on the architecture/ABI of the
  system currently installed (i.e. PYEASYINSTALL_ARCHDEP) the entire version
  string of the system is used in the destination directory, for example;
  
  python_ldap-2.4.12-py2.7-freebsd-10.0-CURRENT-amd64.egg
  
  This plays havoc when STABLE, RELEASE and RELENG are used, not forgetting
  patchlevels as well.  Now, only the major FreeBSD version number is included;
  
  python_ldap-2.4.12-py2.7-freebsd-10-amd64.egg
  
  Given that ABI should be stable at least over major releases, this now allows
  packages to be created and installed correctly.
  
  PR:		ports/133081
  Submitted by:	Stef Walter <stef@memberwebs.com>
  		(this patch used as a starting point)
  Reviewed by:	No objections from python@

Modified:
  head/Mk/bsd.python.mk

Modified: head/Mk/bsd.python.mk
==============================================================================
--- head/Mk/bsd.python.mk	Sat Jun 29 11:26:36 2013	(r322045)
+++ head/Mk/bsd.python.mk	Sat Jun 29 11:43:01 2013	(r322046)
@@ -171,7 +171,7 @@ Python_Include_MAINTAINER=	python@FreeBS
 #
 # PYEASYINSTALL_OSARCH
 #					- Platform identifier for easy_install.
-#					  default: -${OPSYS:L}-${_OSRELEASE}-${ARCH}
+#					  default: -${OPSYS:L}-${OSVERSION:C/([0-9]*)[0-9]{5}/\1/}-${ARCH}
 #							   if PYEASYINSTALL_ARCHDEP is defined.
 #
 # PYEASYINSTALL_CMD - Full file path to easy_install command.
@@ -476,10 +476,8 @@ MAKE_ENV+=						PYTHONPATH=${PYEASYINSTA
 .endif
 
 .if defined(PYEASYINSTALL_ARCHDEP)
-.if !defined(_OSRELEASE)
-_OSRELEASE!=					${UNAME} -r
-.endif
-PYEASYINSTALL_OSARCH?=			-${OPSYS:L}-${_OSRELEASE}-${ARCH}
+PYEASYINSTALL_OSARCH?=			-${OPSYS:L}-${OSVERSION:C/([0-9]*)[0-9]{5}/\1/}-${ARCH}
+MAKE_ENV+=						_PYTHON_HOST_PLATFORM=${PYEASYINSTALL_OSARCH}
 .endif
 PYEASYINSTALL_EGG?=				${PYDISTUTILS_PKGNAME:C/[^A-Za-z0-9.]+/_/g}-${PYDISTUTILS_PKGVERSION:C/[^A-Za-z0-9.]+/_/g}-py${PYTHON_VER}${PYEASYINSTALL_OSARCH}.egg
 PYEASYINSTALL_CMD?=				${LOCALBASE}/bin/easy_install-${PYTHON_VER}
@@ -502,13 +500,13 @@ pre-install-easyinstall:
 
 add-plist-post: add-plist-easyinstall
 add-plist-easyinstall:
-	@# @unexec line must be located before any other line while @exec must not.
-	@${CAT} ${TMPPLIST} > ${TMPPLIST}.pei_tmp
-	@${ECHO_CMD} "@unexec ${PYEASYINSTALL_CMD} ${PYEASYINSTALL_UNINSTALLARGS}" \
-		> ${TMPPLIST}
-	@${CAT} ${TMPPLIST}.pei_tmp >> ${TMPPLIST}
-	@${ECHO_CMD} "@exec ${SETENV} PYTHONPATH=${PYEASYINSTALL_SITELIBDIR} \
-		${PYEASYINSTALL_CMD} ${PYEASYINSTALL_INSTALLARGS}" \
+	@# Easiest to fake pyeasyinstall, or it complains about paths
+	@${ECHO_CMD} "@unexec ${REINPLACE_CMD} -i '' \
+			-e '\,^\./${PYEASYINSTALL_EGG}$$,d' \
+			${PYEASYINSTALL_SITELIBDIR}/easy-install.pth" \
+		>> ${TMPPLIST}
+	@${ECHO_CMD} "@exec ${PRINTF} '1a\n./${PYEASYINSTALL_EGG}\n.\nw\nq\n' | \
+			/bin/ed ${PYEASYINSTALL_SITELIBDIR}/easy-install.pth" \
 		>> ${TMPPLIST}
 .endif		# defined(USE_PYDISTUTILS) && ${USE_PYDISTUTILS} == "easy_install"
 
_______________________________________________
svn-ports-all@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-ports-all
To unsubscribe, send any mail to "svn-ports-all-unsubscribe@freebsd.org"
Comment 10 Chris Rees freebsd_committer 2013-06-29 12:43:51 UTC
State Changed
From-To: open->closed

Hopefully this should work now....