Bug 210820

> *** WARNING: renaming "_ssl" since importing it failed: build/lib.freebsd-11.0-CURRENT-amd64-2.7/_ssl.so: Undefined symbol "SSLv2_method"

https://github.com/yyuu/pyenv/issues/464

=> CFLAGS="-DOPENSSL_NO_SSL2"

(I have not tested it, I can't test it).

(In reply to w.schwarzenfeld from comment #1)
Running "setenv CFLAGS -DOPENSSL_NO_SSL2" prior to running "portupgrade -fprv lang/python27" allowed the build to succeed on my fairly recent 11.0-ALPHA6 VM.

I haven't seen this issue on stable/10.

I tried running that command on my system; however, the build still failed.

(In reply to Gerard Seibert from comment #3)

Does the build still fail due to: Undefined symbol "SSLv2_method"?

Maybe you should first update base, if possible. base/head r296485 is a bit "old".

Comment 5 Fukang Chen 2016-07-10 15:43:05 UTC

*** Bug 210963 has been marked as a duplicate of this bug. ***

Comment 6 Fukang Chen 2016-07-10 17:59:32 UTC

Okay, I will continue posting here.

I think the change was introduced by r417651:
https://svnweb.freebsd.org/ports?view=revision&revision=417651

r417650:
% make WITH_OPENSSL_PORT=1 -C /usr/ports/lang/python27 -V LDFLAGS
 -lpthread -Wl,-rpath,/usr/local/lib -Wl,-rpath=/usr/lib:/usr/local/lib -L/usr/local/lib -fstack-protector

r417651:
% make DEFAULT_VERSIONS+=ssl=openssl -C /usr/ports/lang/python27 -V LDFLAGS
 -lpthread -Wl,-rpath=/usr/lib:/usr/local/lib -Wl,-rpath,/usr/local/lib -fstack-protector

in the previous version, bsd.port.mk loads Mk/bsd.openssl.mk before the USES:
https://svnweb.freebsd.org/ports/head/Mk/bsd.port.mk?view=markup&pathrev=417409#l1354
https://svnweb.freebsd.org/ports/head/Mk/bsd.port.mk?view=markup&pathrev=417409#l1450

Later lang/python27/Makefile has changed to:
USES=          cpe ncurses pkgconfig readline ssl tar:xz

the first "-Wl,-rpath=/usr/lib:/usr/local/lib" comes from Uses/ncurses.mk
then_ssl.so looks for libssl.so.8 in /usr/lib:/usr/local/lib on 11.0-CURRENT,
and the SSLv2 in /usr/lib/libssl.so.8 is disabled there.

# readelf -d /usr/local/lib/python2.7/lib-dynload/_ssl.so | head -n 10

Dynamic section at offset 0x13be0 contains 27 entries:
  Tag        Type                         Name/Value
 0x0000000000000001 (NEEDED)             Shared library: [libthr.so.3]
 0x0000000000000001 (NEEDED)             Shared library: [libssl.so.8]
 0x0000000000000001 (NEEDED)             Shared library: [libcrypto.so.8]
 0x0000000000000001 (NEEDED)             Shared library: [libpython2.7.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libc.so.7]
 0x000000000000000f (RPATH)              Library rpath: [/usr/lib:/usr/local/lib]
 0x000000000000001d (RUNPATH)            Library runpath: [/usr/lib:/usr/local/lib]

I am now finding that ports like Samba are refusing to build because of this problem. Has any progress been made on fixing it?

Thanks :)

Comment 8 Kubilay Kocak 2016-07-15 10:13:33 UTC

CC mat@, who committed (in r417651) the changes mentioned comment 6

(In reply to Gerard Seibert from comment #7)

You could create a Makefile.local in lang/python27.

1. cd /usr/ports/lang/python27
2. Create Makefile.local with this contents:

.if ${OPSYS} == FreeBSD && ${OSVERSION} >= 1100000
CFLAGS+=-DOPENSSL_NO_SSL2
.endif

3. Remove said Makefile.local when it's no longer needed.

I'm at r418574 and lang/python27 still fails to build on amd64 base/head r302830 without the amended CFLAGS.

Comment 10 Fukang Chen 2016-07-15 13:29:49 UTC

Created attachment 172553 [details]
load Uses/ssl.mk before other USES

Here is my temporary workaround to make it load Uses/ssl.mk before other USES.

These ports maybe affected if you compile them with the OpenSSL from the ports on 11.0
http://pastebin.com/raw/CpYyBKpq

(In reply to Trond.Endrestol from comment #9)

That works, the port now builds. Unfortunately, samba43 is now refusing to build; however, I am not sure if that has anything to do with this bug or not.

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=211140

Just a small reminder and more description for those, who want to reproduce this problem (as I came across it yesterday too): The problem is still here and not fixed.

How to reproduce it (as of ports revision r419436, Aug 1 2016):

- Install 11-STABLE/HEAD, where SSLv2 is removed from /usr/lib/libssl.so.8
- Install security/openssl with default options, where SSLv2 in /usr/local/lib/libssl.so.8 is still included
- DO NOT Install or Uninstall port devel/ncurses - then base ncurses are used and -rpath /usr/lib:${LOCALBASE}/lib is added
- Then try to upgrade lang/python27. There is USES=... ncurses ... ssl ..., which implies that cumulative -rpath "/usr/lib:/usr/local/lib:/usr/local/lib" is used, which breaks python's runtime with openssl from ports. More technically, make framework sets LDFLAGS to ... -Wl,-rpath=${NCURSESRPATH} ... ${OPENSSL_LDFLAGS}, where NCURSESRPATH is /usr/lib:${LOCALBASE}/lib and OPENSSL_LDFLAGS is -Wl,-rpath,${OPENSSLRPATH}.

Ultra-fast but Dirty workaround: Swap ncurses and ssl in USES from 
  USES=... ncurses ... ssl ...
to
  USES=... ssl ... ncurses ...
;o)

SSLv2 mismatch (in base it is not, in ports it is) is not needed - but it beautifully reveals, that runtime library used is other than linking library.

Comment 13 Kubilay Kocak 2016-08-05 11:42:38 UTC

(In reply to Fukang Chen from comment #10)

Escalating to portmgr@ (see below)

@portmgr please advise if/what workarounds might be available to python@ to resolve the issue in lang/python* ports until the root cause can be fixed permanently.

@Fukang, can you include that test matrix (pastebin) as an attachment so it doesn't get lost (or missed) please.

Notes:

The CFLAGS modification workaround is not a root-cause fix, as it doesn't resolve the expected/actual runtime library discrepancy (uses base, not ports).

To be clear, it appears thus far from the evidence provided, that this is the result of a conflict caused by the interaction between framework ncurses/ssl bits, relating to base vs ports rpath arguments, exposed by the move from Mk/bsd.foo.mk to Uses/foo.mk. It also appears that this is not strictly, or in principle limited to python, nor ssl and ncurses only, but has a much wider potential for impact.

Q: What other existing base vs ports cases could highlight the same issue?
Q: Does swapping ssl/ncurses in USES not just reverse the problem in the case of base ssl and ports ncurses? If not, why not?

Comment 14 Mathieu Arnold 2016-08-05 12:01:38 UTC

Looking at ncurses.mk, its use of rpath is really wrong, there are three cases, and and it should never add /usr/lib to rpath:

1) USES=ncurses or USES=ncurses:base and devel/ncurses is not present:
use base, no need to add rpath as there is only one libncurses.so present, in /usr/lib
2) USES=ncurses and devel/ncurses is present or USES=ncurses:port:
use ports, and add rpath to LOCALBASE/lib so that the right one is picked
3) USES=ncurses:base and devel/ncurses is present
fail (it would warrant using -rpath /usr/lib, but it error's out during the sanity stage)

I'd say ncurses.mk needs to be patched to not add -rpath in case 1

Comment 15 Mathieu Arnold 2016-08-05 12:03:28 UTC

Created attachment 173310 [details]
ncurses.mk.diff

Something like that

Comment 16 Kubilay Kocak 2016-08-05 12:07:07 UTC

Take back assignment of this issue, in favour of newly created blocking bug 211592 assigned to portmgr.

@Fukang can you please attach your test matrix and proposed patch to bsd.port.mk there please

Comment 17 Kubilay Kocak 2016-08-05 12:07:53 UTC

Any workaround short-term patches to fix the issue in lang/python* ports can be attached here. For any in /usr/port/Mk/, please attach them in bug 211592

Comment 18 Mathieu Arnold 2016-09-29 14:47:35 UTC

Unless mistaken, this was fixed in r421554.

I installed FreeBSD 11.0-RELEASE along with the ports collection from the disc1 CD

/etc/make.conf has "WITH_OPENSSL_PORT=YES" and "DEFAULT_VERSIONS+=ssl=openssl"

I installed the security/openssl port & then tried to install lang/python2.7 only to experience the same failure as described herein.

My horrid kluge was to "touch /usr/ports/lang/python27/work/stage/usr/local/lib/python2.7/lib-dynload/_ssl.so" which allowed things to continue, although it's probably an accident waiting to happen ...