Bug 242896
| Summary: | lang/python*: Fail to package in poudriere (testport) when cross-building | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Product: | Ports & Packages | Reporter: | Christian Ullrich <chris> | ||||||
| Component: | Individual Port(s) | Assignee: | freebsd-python (Nobody) <python> | ||||||
| Status: | Open --- | ||||||||
| Severity: | Affects Some People | CC: | bdrewery, koobs, mat, pi, python | ||||||
| Priority: | --- | Keywords: | needs-qa | ||||||
| Version: | Latest | Flags: | koobs:
merge-quarterly?
|
||||||
| Hardware: | Any | ||||||||
| OS: | Any | ||||||||
| See Also: | https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=208282 | ||||||||
| Attachments: |
|
||||||||
Thank you for the report Christian, ^Triage: Request feedback from committer (sbruno) who resolved bug 208282 The block in question added in ports r447129 probably needs to be done earlier (than post-install) Created attachment 210252 [details]
Patch.
This patch fixes this bug and two related ones.
Firstly, it recompiles the .pyc files after modifying the .py file. This applies to lang/python3[578] because lang/python36 already has (most of) it.
Secondly, it adds the missing recompilation with -OO to lang/python36.
Thirdly, it generates correct file names involving ${ABIFLAGS}. There are several places in post-install where file names are assembled that include ABI flags. Previously, these were fixed as "m" (PYMALLOC enabled) for 3.[67] and empty for 3.8 (where the "m" flag has been removed upstream). (This only fixes the clearly somewhat rare case of cross-building 3.[67] with native-xtools and without PYMALLOC.)
Tested lang/python3[5-8] using poudriere bulk -t cross-building to aarch64, with and without PYMALLOC. Once the packages were successfully built, I manually checked that the .pycs are newer than the .py and that neither contains the /nxb-bin prefix.
Tested devel/py-zope.interface@py3[5-8] using poudriere bulk -t cross-building to aarch64, with and without PYMALLOC, using BUILD_ALL_PYTHON_FLAVORS. This port includes a binary extension, for added ABIFLAGS enjoyment. (I noticed that this .so is linked to the respective libpython....so with 3.[567], but not with 3.8. It can be imported and calling random functions on it does not crash, so I assume this is by design.)
I don't think I'll be able to look into this so I'm hoping someone else can pick this up. Someone with a better understanding of the machine that is the python build system would be desirable. Created attachment 215000 [details]
Patch.
Rebased patch after r536770.
|
[This is difficult to explain, and I'm not sure I've entirely understood it myself. Bear with me, please.] "poudriere testport" (as well as bulk -t) of Python ports (not the lang/python*, but the py-* ports) fails during stage when using a cross-build jail with native-xtools (/nxb-bin): =>> Error: Filesystem touched during stage (files must install to ${STAGEDIR}): usr/local/lib/python3.7/__pycache__/_sysconfigdata_m_freebsd12_.cpython-37.pyc: size (18800, 18752) Bug 208282 introduced a fix for incorrect contents of _sysconfigdata_etc_.py in the cross-build situation by SUB_FILESing away the /nxb-bin prefix, which has apparently worked well since, and *does* work (albeit by accident, see below) with "poudriere bulk". It looks like there was an oversight in this fix, in that the .py file is rewritten *after* it has been byte-compiled. Hence, the .pyc files still contain the /nxb-bin prefix in the various variables: # tar -xvf /usr/local/poudriere/data/packages/aarch64-default-raspi3/All/python37-3.7.6.txz &> /dev/null # fgrep nxb usr/local/lib/python3.7/_sysconfigdata_m_freebsd12_.py # strings usr/local/lib/python3.7/__pycache__/_sysconfigdata_m_freebsd12_.cpython-37.pyc | fgrep nxb-bin/usr/bin/cc | head -n 1 -L. -lpython3.7mzO/nxb-bin/usr/bin/cc -pthread -shared -L/usr/local/lib -fstack-protector-strongZ This inconsistency does not affect poudriere bulk because poudriere only looks for staging violations in testport, and in bulk with -t. However, the .pyc is updated during the build once something imports sysconfig, leading to the staging violation. I have no clue how to correctly fix this. Perhaps by recompiling _sysconfigdata_etc_.py after removing the prefixes, so that a) the files are consistent and b) the .pyc files' mtime is updated avoiding the automatic recompilation?