Bug 168640 - Mk/Uses/python.mk Prevent checksum mismatches for compiled python files on installing python ports via the ports tree
Summary: Mk/Uses/python.mk Prevent checksum mismatches for compiled python files on in...
Status: In Progress
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Ports Framework (show other bugs)
Version: Latest
Hardware: Any Any
: Normal Affects Only Me
Assignee: freebsd-python mailing list
URL:
Keywords: needs-qa
Depends on:
Blocks:
 
Reported: 2012-06-03 16:00 UTC by Marcus von Appen
Modified: 2015-01-02 03:25 UTC (History)
1 user (show)

See Also:


Attachments
file.diff (506 bytes, patch)
2012-06-03 16:00 UTC, Marcus von Appen
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Marcus von Appen freebsd_committer freebsd_triage 2012-06-03 16:00:26 UTC
        The default configuration within bsd.python.mk installs python
        packages that use distutils or easy_install with byte-compiled
        files.

        The compiled (.pyc/.pyo) files contain a timestamp and magic
        number which are used as identicators for recompiling the files,
        if their original source (.py) differs in its st_mtime value or
        if Python's magic number changes.

        Whenever the byte-compiled files change, a checksum mismatch
        will be raised on deinstall or reinstalling the port.

        For a lengthy rant about this issue, check
            http://wiki.freebsd.org/Ports/Python/CompiledPackages

Fix: The following fix introduces a new _CLEANUP_CONTENTS command
        chain, which is injected into the fake-pkg: target. It can be
        used to perform clean-up operations on the +CONTENTS file of
        the pkg_install toolchain.

        Within bsd.python.mk _CLEANUP_CONTENTS will be enhanced, so that
        the @comment MD5: entry for +CONTENTS entries, that denote a
        .pyc or .py file, is removed.

        The change will need an -exp run to verify that the installation
        and deinstallation still works for all ports.


+_CLEANUP_CONTENTS?=	${DO_NADA};
+
 # Fake installation of package so that user can pkg_delete it later.
 # Also, make sure that an installed port is recognized correctly in
 # accordance to the @pkgdep directive in the packing lists
@@ -5869,6 +5871,7 @@
 		${ECHO_MSG} "===>   Registering installation for ${PKGNAME}"; \
 		${MKDIR} ${PKG_DBDIR}/${PKGNAME}; \
 		${PKG_CMD} ${PKG_ARGS} -O ${PKGFILE} > ${PKG_DBDIR}/${PKGNAME}/+CONTENTS; \
+		${_CLEANUP_CONTENTS} \
 		${CP} ${DESCR} ${PKG_DBDIR}/${PKGNAME}/+DESC; \
 		${ECHO_CMD} ${COMMENT:Q} > ${PKG_DBDIR}/${PKGNAME}/+COMMENT; \
 		if [ -f ${PKGINSTALL} ]; then \


+# Override bsd.port.mk's setting
+_CLEANUP_CONTENTS+=	\
+	${SED} -i -e '/\.py[co]$$/{n;d;}' ${PKG_DBDIR}/${PKGNAME}/+CONTENTS;
+
 .if !defined(PYDISTUTILS_NOEGGINFO) && \
 	(defined(INSTALLS_EGGINFO) ||	\
 		(defined(USE_PYDISTUTILS) && \--HMr61fieeubDChPaotewyQR8C0yK6gxBv7VPAd2QDphqSWN6
Content-Type: text/plain; name="file.diff"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment; filename="file.diff"

Index: Mk/bsd.port.mk
===================================================================
RCS file: /home/pcvs/ports/Mk/bsd.port.mk,v
retrieving revision 1.724
diff -u -r1.724 bsd.port.mk
--- Mk/bsd.port.mk	31 May 2012 14:27:23 -0000	1.724
+++ Mk/bsd.port.mk	3 Jun 2012 12:24:32 -0000
@@ -5850,6 +5850,8 @@
 .endif
 .endif
How-To-Repeat: 	- install a python port that installs .pyc/.pyo files
        - slightly modify one of the original .py files - OR -
          - change the date/timestamp on the original file (using touch)
          - deinstall/reinstall the Python interpreter
        - (as root user) import the file as module in the python interpreter
        - deinstall the port
Comment 1 Ruslan Makhmatkhanov freebsd_committer 2012-06-03 17:47:35 UTC
Responsible Changed
From-To: freebsd-ports-bugs->freebsd-python

Python pr
Comment 2 Marcus von Appen freebsd_committer freebsd_triage 2012-07-20 21:34:13 UTC
Responsible Changed
From-To: freebsd-python->portmgr

Over to portmgr, since bsd.port.mk is portmgr territory. An exp-run will 
be needed to ensure that nothing breaks.
Comment 3 Bryan Drewery freebsd_committer 2013-10-09 12:21:39 UTC
Is this still valid?

-- 
Regards,
Bryan Drewery
Comment 4 Marcus von Appen freebsd_committer freebsd_triage 2013-10-10 06:45:22 UTC
On, Wed Oct 09, 2013, Bryan Drewery wrote:

> Is this still valid?

Yes. The problem still exists.

Cheers
Marcus
Comment 5 Bryan Drewery freebsd_committer 2013-10-10 13:05:15 UTC
On 10/10/2013 12:45 AM, Marcus von Appen wrote:
> On, Wed Oct 09, 2013, Bryan Drewery wrote:
> 
>> Is this still valid?
> 
> Yes. The problem still exists.


Right, but is the PATCH still valid and sufficient? I recall some effort
to fix this previously.


-- 
Regards,
Bryan Drewery
Comment 6 Marcus von Appen freebsd_committer freebsd_triage 2013-10-23 10:18:54 UTC
Bryan Drewery <bdrewery@freebsd.org>:

> On 10/10/2013 12:45 AM, Marcus von Appen wrote:
>> On, Wed Oct 09, 2013, Bryan Drewery wrote:
>>
>>> Is this still valid?
>>
>> Yes. The problem still exists.
>
> Right, but is the PATCH still valid and sufficient? I recall some effort
> to fix this previously.

I need to check, if the patch still cleanly applies (I doubt that) and  
I am not
satisfied with its hackish approach. A cleaner way (e.g. with an internal
plist-post-cleanup: hook) would be nice, so it can be reused multiple times
throughout different .mk files and (maybe) even used by some ports,  
which might
create variable content files, that could create checksum mismatches.

Would such a hook be desirable for portmgr or should this exist (for now) as a
python-specific hack? In either case, I'll recheck and update the patch.

Cheers
Marcus
Comment 7 Marcus von Appen freebsd_committer freebsd_triage 2013-11-03 14:12:08 UTC
Responsible Changed
From-To: portmgr->freebsd-python

Back to python@, we should evaluate the patch once more before doing 
anything
Comment 8 Carlo Strub freebsd_committer 2014-08-31 17:24:57 UTC
Is this bug still relevant?
Comment 9 Marcus von Appen freebsd_committer freebsd_triage 2014-08-31 18:28:30 UTC
Yes
Comment 10 Kubilay Kocak freebsd_committer freebsd_triage 2014-09-05 15:19:52 UTC
Canonicalise python@ assignment.