Bug 211592 - bsd.ssl.mk migration to USES exposed conflict between ncurses/ssl rpaths (base vs ports)
Summary: bsd.ssl.mk migration to USES exposed conflict between ncurses/ssl rpaths (bas...
Status: Closed FIXED
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Ports Framework (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Many People
Assignee: Port Management Team
URL:
Keywords: needs-qa, patch, regression
Depends on:
Blocks: 210820
  Show dependency treegraph
 
Reported: 2016-08-05 12:05 UTC by Kubilay Kocak
Modified: 2016-09-11 10:40 UTC (History)
2 users (show)

See Also:
koobs: merge-quarterly?


Attachments
LDFLAGS.txt (9.46 KB, text/plain)
2016-08-06 07:02 UTC, Fukang Chen
no flags Details
bsd.port.mk.diff (468 bytes, text/plain)
2016-08-06 07:52 UTC, Fukang Chen
no flags Details
patch (1.59 KB, patch)
2016-08-10 12:14 UTC, Mathieu Arnold
no flags Details | Diff
gssapi.mk.diff (363 bytes, text/plain)
2016-08-11 12:30 UTC, Fukang Chen
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Kubilay Kocak freebsd_committer freebsd_triage 2016-08-05 12:05:25 UTC
r417651 migrated bsd.ssl.mk to USES, exposing a conflict between rpath arguments provided by Uses/ncurses.mk and Uses/ssl.mk.

This results in build time and runtime library linking discrepancies in certain common circumstances such as, but not limited to using ncurses from base, but openssl from ports, and a failure in at least lang/python27, reported in bug 210820

A test matrix of the difference in LDFLAG values for various ports between r417651 and r417650 (the tree before the commit) and patch has been provided for bsd.port.mk in bug 210820 that will be attached here instead.
Comment 1 Fukang Chen freebsd_committer 2016-08-06 07:02:37 UTC
Created attachment 173350 [details]
LDFLAGS.txt

Here's a list of make -V LDFLAGS output, these ports might be affected by loading ssl.mk after other USES.

I just noticed there's a port security/heimdal that doesn't have ncurses in USES:

r417650: -Wl,-rpath,/usr/local/lib/heimdal:/usr/local/lib   -L/usr/local/lib
r417651:  -Wl,-rpath,/usr/local/lib/heimdal:/usr/lib -L/usr/local/lib -Wl,-rpath,/usr/local/lib

LDFLAGS is empty when security/heimdal loads gssapi.mk,
then it goes to line 162:

https://svnweb.freebsd.org/ports/head/Mk/Uses/gssapi.mk?view=markup#l156
156 	.if !empty(LDFLAGS:M-Wl,-rpath,*)
157 	.for F in ${LDFLAGS:M-Wl,-rpath,*}
158 	LDFLAGS:=       -Wl,-rpath,${_RPATH}:${F:S/-Wl,-rpath,//} \
159 	                ${LDFLAGS:N-Wl,-rpath,*}
160 	.endfor
161 	.else
162 	LDFLAGS+=       -Wl,-rpath,${_RPATH}:/usr/lib
163 	.endif
Comment 2 Fukang Chen freebsd_committer 2016-08-06 07:52:36 UTC
Created attachment 173351 [details]
bsd.port.mk.diff

Here is my temporary workaround I used these days to make it load Uses/ssl.mk before other USES. I'm not proposing it as the fix. 

Maybe we can come up something to cacluate the load sequence automatically like rcorder(8), or the Uses/*.mk should have no dependency ordering at all ...
Comment 3 Mathieu Arnold freebsd_committer 2016-08-06 08:12:55 UTC
This change is wrong, the order into which USES are loaded should not change the outcome. From what I read, ncurses.mk is the one doing bad things here.
Comment 4 Fukang Chen freebsd_committer 2016-08-06 09:28:59 UTC
Please correct me if I'm wrong, I think the settings in the Uses/*.mk are appended by
the order they loaded.

r417650 loads bsd.ssl.mk first then Uses/ncurses.mk:
% make -C /usr/ports/lang/python27 -dv -V PKGNAME 2>&1 | grep Global:LDFLAGS
Global:LDFLAGS = 
Global:LDFLAGS =  ${${:UTHREADS}_${:ULDFLAGS}}
Global:LDFLAGS =  ${${:UTHREADS}_${:ULDFLAGS}} ${OPENSSL_LDFLAGS}
Global:LDFLAGS =  ${${:UTHREADS}_${:ULDFLAGS}} ${OPENSSL_LDFLAGS} -Wl,-rpath=${NCURSESRPATH}
Global:LDFLAGS =  ${${:UTHREADS}_${:ULDFLAGS}} ${OPENSSL_LDFLAGS} -Wl,-rpath=${NCURSESRPATH} -L${LOCALBASE}/lib

r417651 loads Uses/ssl.mk first then Uses/ncurses.mk:
% make -C /usr/ports/lang/python27 -dv -V PKGNAME 2>&1 | grep Global:LDFLAGS
Global:LDFLAGS = 
Global:LDFLAGS =  ${${:UTHREADS}_${:ULDFLAGS}}
Global:LDFLAGS =  ${${:UTHREADS}_${:ULDFLAGS}} -Wl,-rpath=${NCURSESRPATH}
Global:LDFLAGS =  ${${:UTHREADS}_${:ULDFLAGS}} -Wl,-rpath=${NCURSESRPATH} -L${LOCALBASE}/lib
Global:LDFLAGS =  ${${:UTHREADS}_${:ULDFLAGS}} -Wl,-rpath=${NCURSESRPATH} -L${LOCALBASE}/lib ${OPENSSL_LDFLAGS}

for the port security/heimdal, gssapi.mk is loaded first, 
when it sees LDFLAGS empty, LDFLAGS is set to -Wl,-rpath,${_RPATH}:/usr/lib

r417650 loads bsd.ssl.mk first then Uses/gssapi.mk
% make -C /usr/ports/security/heimdal -dv -V PKGNAME 2>&1 | grep Global:LDFLAGS
Global:LDFLAGS = 
Global:LDFLAGS =  ${OPENSSL_LDFLAGS}
Global:LDFLAGS = -Wl,-rpath,/usr/local/lib/heimdal:/usr/local/lib  
Global:LDFLAGS = -Wl,-rpath,/usr/local/lib/heimdal:/usr/local/lib   -L${LOCALBASE}/lib

r417651 loads Uses/gssapi.mk then Uses/ssl.mk
% make -C /usr/ports/security/heimdal -dv -V PKGNAME 2>&1 | grep Global:LDFLAGS
Global:LDFLAGS = 
Global:LDFLAGS =  -Wl,-rpath,${_RPATH}:/usr/lib
Global:LDFLAGS =  -Wl,-rpath,${_RPATH}:/usr/lib -L${LOCALBASE}/lib
Global:LDFLAGS =  -Wl,-rpath,${_RPATH}:/usr/lib -L${LOCALBASE}/lib ${OPENSSL_LDFLAGS}
Comment 5 Fukang Chen freebsd_committer 2016-08-06 09:32:53 UTC
(In reply to Fukang Chen from comment #4)
> r417651 loads Uses/ssl.mk first then Uses/ncurses.mk:
sorry, it should be 
r417651 loads Uses/ncurses.mk first then Uses/ssl.mk
Comment 6 Mathieu Arnold freebsd_committer 2016-08-10 12:14:20 UTC
Created attachment 173494 [details]
patch

There are three cases here:

1) USES=ncurses or USES=ncurses:base and devel/ncurses is not present:
use base, no need to add rpath as there is only one libncurses.so present, in /usr/lib
2) USES=ncurses and devel/ncurses is present or USES=ncurses:port:
use ports, and add rpath to LOCALBASE/lib so that the right one is picked
3) USES=ncurses:base and devel/ncurses is present
fail (it would warrant using -rpath /usr/lib, but it error's out during the sanity stage)
Comment 7 Fukang Chen freebsd_committer 2016-08-11 12:30:14 UTC
Created attachment 173545 [details]
gssapi.mk.diff

Hi Mathieu,

Thanks for the nucrses.mk patch. I think it should fixed all the ports that have ncurses in USES.
I just tested it with lang/python27, it worked. :)

gssapi.mk has a similar problem. Here are steps to reproduce it.
compiling mail/fetchmail with the GSSAPI_HEIMDAL option
and it uses the OpenSSL from base.

# cd /usr/ports/mail/fetchmail
# make BATCH=1 WITHOUT+=GSSAPI_BASE WITH+=GSSAPI_HEIMDAL DEFAULT_VERSIONS+=ssl=openssl

# ldd work/stage/usr/local/bin/fetchmail | grep ssl.so
   libssl.so.8 => /usr/lib/libssl.so.8 (0x20188000)

# readelf -d work/stage/usr/local/bin/fetchmail | grep PATH
 0x0000000f RPATH                Library rpath: [/usr/local/lib/heimdal:/usr/lib:/usr/local/lib]
 0x0000001d RUNPATH              Library runpath: [/usr/local/lib/heimdal:/usr/lib:/usr/local/lib]

# make BATCH=1 WITHOUT+=GSSAPI_BASE WITH+=GSSAPI_HEIMDAL DEFAULT_VERSIONS+=ssl=openssl -V LDFLAGS
 -L/usr/local/lib -Wl,-rpath,/usr/local/lib/heimdal:/usr/lib -Wl,-rpath,/usr/local/lib
Comment 8 Mathieu Arnold freebsd_committer 2016-08-11 20:45:37 UTC
Mmmm, yes, gssapi.mk needs to be fixed to remove /usr/lib too.

There's never a good reason for putting it in.  For GSSAPI, there are two cases:

1) OpenSSL from ports is installed
   -> Any GSSAPI from ports, but can't have base.
2) OpenSSL from the base system
   -> Any GSSAPI you want
Comment 9 commit-hook freebsd_committer 2016-09-08 14:13:02 UTC
A commit references this bug:

Author: mat
Date: Thu Sep  8 14:12:56 UTC 2016
New revision: 421554
URL: https://svnweb.freebsd.org/changeset/ports/421554

Log:
  Fix ncurses to not add /usr/lib to rpath.

  There are three cases here:

  1) USES=ncurses or USES=ncurses:base and devel/ncurses is not present:
     use base, no need to add rpath as there is only one libncurses.so
     present, in /usr/lib
  2) USES=ncurses and devel/ncurses is present or USES=ncurses:port: use
     ports, and add rpath to LOCALBASE/lib so that the right one is picked
  3) USES=ncurses:base and devel/ncurses is present fail (it would warrant
     using -rpath /usr/lib, but it error's out during the sanity stage)

  PR:		211592
  Reported by:	koobs (and many others)
  Reviewed by:	bapt
  Sponsored by:	Absolight
  Differential Revision:	https://reviews.freebsd.org/D7459

Changes:
  head/Mk/Uses/ncurses.mk