Created attachment 190911 [details] Build logs, env and ver. details, config options. Previous commit builds fine on both amd64 and powerpc. Today's r462630 commit won't build/install packages on both archs: 11.1-RELEASE; pkg-static: Unable to access file /usr/ports/lang/python27/work/stage/usr/local/lib/python2.7/lib-dynload/_elementtree.so:No such file or directory pkg-static: Unable to access file /usr/ports/lang/python27/work/stage/usr/local/lib/python2.7/lib-dynload/pyexpat.so:No such file or directory Attached is the build log, uname, pkg version -v, etc. in a single file.
This will likely be related to the CFLAGS/LIBS change (additions) in ports r462630 For more context see this block in python27/Makefile: # Use CPPFLAGS over CFLAGS due to -I ordering, causing elementtree and pyexpat # to break in Python 2.7, or preprocessor complaints in Python >= 3.3 # Upstream Issue: https://bugs.python.org/issue6299 The issue is not scoped only to NLS. For more context see: https://bugs.python.org/issue6299#msg210189 There have been countless ports issues reported in the past related to hacking/modifying CFLAGS (and other variables, and these changes should not be attempted until the root underlying issue(s) are fixed (correctly/completely). I'll see if I can prepare a patch for you to test. If that can't be done, we'll need to revert the change.
I cannot reproduce the failure in poudriere builds (11.1-RELEASE amd64 with the specific option setting: -DEBUG +IPV6 +LIBFFI +NLS +PYMALLOC +THREADS -UCS2 +UCS4).
(In reply to Sunpoet Po-Chuan Hsieh from comment #2) Likewise. @Chip Can you provide system environment/configuration information (uname -a, sanitized /etc/make.conf) along with an the full build log (as an attachmennt)
@Sunpoet, Can we hold off on MFH'ing ports r462630 until we figure out whats happening?
(In reply to Kubilay Kocak from comment #3) @Kubilay there should be an attachment already with my build log, env/config details etc. However here's my make.conf FAVORITE_COMPILER=clang COMPILER_TYPE= clang MAKE_JOBS_NUMBER?=3 CC=clang CXX=clang++ CPP=clang-cpp WITH_CCACHE_BUILD=yes CPUTYPE?=core2 DEFAULT_VERSIONS+=ssl=openssl OPENSSLBASE=/usr/local # Use ports version of openssl with TLS 1.2 OPTIONS_UNSET+= GSSAPI_BASE OPTIONS_SET+= GSSAPI_MIT Thanks!
(In reply to Chip Cuccio from comment #5) Hi Chip, Please remove > CC=clang > CXX=clang++ > CPP=clang-cpp > OPENSSLBASE=/usr/local from your make.conf, these are redundant and set either by Mk/Uses/ssl.mk or in the ports framework.
(In reply to Bernard Spil from comment #6) @Bernard, OK I will do that, and report back. Thanks! - chip
Created attachment 190956 [details] Updated build log after make.conf changes.
(In reply to Bernard Spil from comment #6) @Bernard, I have made the make.conf changes as you advised: Same build issue. Updated build log attached. Thanks!
(In reply to Kubilay Kocak from comment #4) Yes, MFH is suspended.
Here is my poudriere build log of lang/python27-pr226135 (copied from lang/python27 with following change). https://people.freebsd.org/~sunpoet/logs/python27-pr226135.log % diff -u lang/python27/Makefile lang/python27-pr226135/Makefile --- lang/python27/Makefile 2018-02-23 03:51:29.048434000 +0800 +++ lang/python27-pr226135/Makefile 2018-02-24 01:22:44.675219000 +0800 @@ -5,7 +5,8 @@ PORTREVISION= 1 CATEGORIES= lang python ipv6 MASTER_SITES= PYTHON/ftp/python/${PORTVERSION} -PKGNAMESUFFIX= 27 +#PKGNAMESUFFIX= 27 +PKGNAMESUFFIX= 27-pr226135 DISTNAME= Python-${PORTVERSION} DIST_SUBDIR= python @@ -44,7 +45,8 @@ OSMAJOR=${OSVERSION:C/([0-9]*)[0-9]{5}/\1/} # For plat-freebsd* in pkg-plist. https://bugs.python.org/issue19554 OPTIONS_DEFINE= DEBUG IPV6 LIBFFI NLS PYMALLOC THREADS -OPTIONS_DEFAULT= LIBFFI PYMALLOC THREADS UCS4 +#OPTIONS_DEFAULT= LIBFFI PYMALLOC THREADS UCS4 +OPTIONS_DEFAULT= LIBFFI NLS PYMALLOC THREADS UCS4 OPTIONS_SINGLE= UNICODE OPTIONS_SINGLE_UNICODE= UCS2 UCS4 OPTIONS_SUB= yes
@Chip Could you please: - Comment out the LIBS entry - Rebuild, confirm whether issue is still reproducible - In either case, attach the build log
@Chip Additionally, could you please run a second test: Using python27 *as is* (verbatim r462630, *without* requested change in comment 10) - pkg delete -f expat - rebuild python27 - confirm whether issue is reproducible I'm continuing to isolate locally
The issue is reproducible with: - DEFAULT_VERSIONS=ssl=openssl (more precisely CFLAGS+=-I/usr/local/include) - expat2 port/package installed The root cause is that described in comment #1 I'll have a patch shortly
Created attachment 190984 [details] Use CPPFLAGS, Make CFLAGS comment more prominent @Chip Please test/confirm that this patch fixes the issue.
(In reply to Kubilay Kocak from comment #15) @Koobs, I can confirm that your patch corrects the build issue on both amd64 and powerpc archs! Thank you. - Chip
(In reply to Kubilay Kocak from comment #15) This patch looks good to me. And it's reasonable for CFLAGS/CPPFLAGS issue you mentioned before. I'll check if it works with 3 SSL settings (ssl=base, openssl and openssl-devel). Thanks!
I had the same issue on FreeBSD 10.3-RELEASE-p26 #0 r327588 building python27-2.7.14_1, applied the patch and build completed successfully. Thank you for the patch.
I've got the same problem with port build (latest 11.1 amd64) Also can confirm: patch from comment #15 was helpful, thanks. Hope it will be committed to ports soon.
@koobs, I've tested it with different SSL settings. It works. Please DO NOT BUMP PORTREVISION. It fixes the build but does not change the package, especially for people who did not encounter build failure.
*** Bug 226256 has been marked as a duplicate of this bug. ***
Why not back out the old commit? It has been failing for over a week now.
(In reply to Sunpoet Po-Chuan Hsieh from comment #20) I note that other python ports didn't get the original change (ports r462630, as part of bug 222795), but also: - python34 still has BROKEN_SSL=openssl-devel - python35 doesn't - python36 doesn't Were you going to apply r462630 to the other ports as well? What is the status with regard to ssl support in other ports? (>2.7) Barnerd, if you can comment on the change delta required to get all lang/python?? ports to support openssl/openssl-devel/libressl, that would be appreciated. We (python) want as much consistency across the ports as possible, so I'll want to replicate this change into every port unless there are reasons not to.
(In reply to Kubilay Kocak from comment #23) Other python port do not need r462630 as follows: - python34 is confirmed broken with/without r462630. - python35 doesn't need r462630. - python36 doesn't need r462630.
A commit references this bug: Author: koobs Date: Thu Mar 1 13:47:11 UTC 2018 New revision: 463321 URL: https://svnweb.freebsd.org/changeset/ports/463321 Log: lang/python27: Fix build regression r462630 added OpenSSL include/library paths to CFLAGS/LIBS which caused the _elementtree and pyexpat modules to fail to build. Adding paths to CFLAGS is known to cause issues [1]. Accordingly: - Switch to using CPPFLAGS for OpenSSL include/library paths (like NLS). - Move the comment describing why this is necessary to the top of the port with reasonable warning about what can happen and why. [1] https://svnweb.freebsd.org/ports?view=revision&revision=326729 PR: 226135, 222795) Reported by: many Reviewed by: sunpoet (python) Approved by: koobs (python) Changes: head/lang/python27/Makefile
(In reply to Sunpoet Po-Chuan Hsieh from comment #24) Thank you. Anything left to do here?
(In reply to Kubilay Kocak from comment #26) Well done. Thank you!
yes, thank you!
A commit references this bug: Author: antoine Date: Fri Mar 2 06:54:14 UTC 2018 New revision: 463375 URL: https://svnweb.freebsd.org/changeset/ports/463375 Log: Revert r462630, it introduced regressions PR: 226135 With hat: portmgr Changes: head/lang/python27/Makefile
Thank you Antoine. All: ports r463321 (comment 25), fixing the regression was reverted due to a secondary readline build failure on what appears to be 10.x, so I asked for r462630 (the original regression) to be reverted as it was wider in scope.
*** Bug 226321 has been marked as a duplicate of this bug. ***
I'm not sure, that my https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=226321 is duplicate of this one (I tried reverted patch from here) as failure occurs only with libressl, but OK, let it be.
Only for information: There is python 2.7.15 with fixes for at least CVE-2018-1060 and CVE-2018-1061.
The original bug report (regression) is resolved. Re comment 33, see bug 228028