Bug 206194

Summary: lang/python27 install fails if misc/compat6x is installed [due to libreadline check]
Product: Ports & Packages Reporter: Harald Schmalzbauer <bugzilla.freebsd>
Component: Ports FrameworkAssignee: Mark Linimon <linimon>
Status: Closed Not A Bug    
Severity: Affects Some People CC: brent, python
Priority: --- Keywords: needs-qa
Version: Latest   
Hardware: Any   
OS: Any   
Attachments:
Description Flags
python27 build log none

Description Harald Schmalzbauer 2016-01-13 11:34:21 UTC 
In my case, trying to install lang/python27 (needed for x11/libxcb:Makefile USES=python:build) failed with:
pkg-static: Unable to access file /usr/local/ports-wrktree/lang/python27/work/stage/usr/local/lib/python2.7/lib-dynload/readline.so

_Cause:_ misc/compat6x

Illustration:
lang/python27/Makefile has "USES= … readline …"

FreeBSD ports system checks:
=> SHA256 Checksum OK for python/Python-2.7.11.tar.xz.
…
===>   python27-2.7.11_1 depends on shared library: libreadline.so.6 - found (/usr/local/lib/compat/libreadline.so.6)

and finds libreadline.so.6 in /usr/local/lib/compat/.

Now python's build check has different results:
checking how to link readline libs... none
checking for rl_callback_handler_install in -lreadline... no
checking for rl_pre_input_hook in -lreadline... no
checking for rl_completion_display_matches_hook in -lreadline... no
checking for rl_completion_matches in -lreadline... no

which leads to the following warning during build:
Python build finished, but the necessary bits to build these modules were not found:
_bsddb             _sqlite3           _tkinter        
dl                 gdbm               imageop         
linuxaudiodev      readline           spwd            
sunaudiodev                                           
To find the necessary bits, look in setup.py in detect_modules() for the module's name.

So lib-dynload/readline.so wasn't built, but pkg-plist insists on registering it, hence the above quoted failure message.

_Workaround/Solution:_
cd /usr/ports/devel/readline && make install …

_before_ building lang/python27.

The resulting difference is:
===>   python27-2.7.11_1 depends on shared library: libreadline.so.6 - found (/usr/local/lib/libreadline.so.6)
leading to:
checking how to link readline libs... -lreadline
checking for rl_callback_handler_install in -lreadline... yes
checking for rl_pre_input_hook in -lreadline... yes
checking for rl_completion_display_matches_hook in -lreadline... yes
checking for rl_completion_matches in -lreadline... yes


Most likely uninstalling compat6x also is a workaround.

Real solution was to make ports/Mk/Uses/readline.mk misc/compat aware! But unfortunately I don't have a single spare minute to look into it :-(
Comment 1 Kubilay Kocak freebsd_committer freebsd_triage 2016-01-13 11:55:37 UTC 
This is interesting. I'll pose some questions (somewhat rhetorically) to help us determine if, how and where this issue might be best addressed.

* If compatNx is installed, does that mean the user *always* wants or expects *everything* thing to link against compat libraries? There is a user answer to this question and potentially a ports framework answer, which may be different.

* Given the current description of the problem, I would expect *many* ports to be incorrectly linking against or not finding the expected compat libraries, in this case libreadline via USES. Do we know of any other cases that: a) Fail in a similar way (not found, whether they fail to package or not), b) Fail in other ways (linking to different library than expected)

* How will readline.mk decide when to depend on compatNx libraries, rather than either existing base ones, or ones from ports, and on what basis and level of granularity.


* Does, and if so how is this addressed for other libraries provided in compatNx's that have ports framework and individual port implications?

* What is/are the actual 'root' cause(s) here? 

lang/python27 is implicated only in so far as it doesn't check whether certain C extensions compiled or not, before adding them to the plist. This 'could' be fixed. This is a fairly common type of failure across many ports, whether by virtue of a compile failure, or an OPTION.

USES=readline is implicated only in so far as it does not detect the presence of compatNx readline library and provide the necessary *FLAGS to find it.
Comment 2 Harald Schmalzbauer 2016-01-13 12:19:16 UTC 
(In reply to Kubilay Kocak from comment #1)

I think your question, when to link against compatXx libs, or better to solve "native" dependency, is the most important to figure out.
My first idea was to implement interactivity to ask, if it's no batch-build. But that's probably not consumer friendly and a qualified answer is not that easy either; at least for me.

I'm not aware of any other ports linking against the "wrong" libreadline, but I'm not frequently building/checking readline-dependent ports.
My first action was to delay compat6x to be installed on building machines for now!

Thanks for your quick attention, hope at least the report helps, won't be able to help out more.

-Harry
Comment 3 Mathieu Arnold freebsd_committer freebsd_triage 2016-04-13 11:35:34 UTC 
moving that to python@
Comment 4 Brent Busby 2017-08-06 03:56:17 UTC 
I'm still experiencing this problem on FreeBSD 10.4-PRERELEASE in a jail during the make install phase.  devel/readline is installed before install of python27 was attempted, and compat6x is not installed either as a port/package nor is COMPAT_FREEBSD6 in the kernel.  The base machine install and the jail are fresh and up to date tracking stable as of today, and all packages are up to date as well.


===>  Installing for python27-2.7.13_7
===>   python27-2.7.13_7 depends on shared library: libffi.so - found (/usr/local/lib/libffi.so)
===>   python27-2.7.13_7 depends on shared library: libreadline.so.7 - found (/lib/libreadline.so.7)
===>   python27-2.7.13_7 depends on shared library: libintl.so - found (/usr/local/lib/libintl.so)
===>  Checking if python27 already installed
actual-package-depends: dependency on /lib/libreadline.so.7 not registered (normal if it belongs to base)
===>   Registering installation for python27-2.7.13_7
pkg-static: Unable to access file /var/ports/usr/ports/lang/python27/work/stage/usr/local/lib/python2.7/lib-dynload/readline.so:No such file or directory
*** Error code 74

Stop.
make[1]: stopped in /usr/ports/lang/python27
*** Error code 1

Stop.
make: stopped in /usr/ports/lang/python27
Comment 5 Kubilay Kocak freebsd_committer freebsd_triage 2017-08-06 04:26:56 UTC 
(In reply to Brent Busby from comment #4)

Your issue may be an unrelated failure, particularly if compat6x (that this report relies on) is not installed.

What we need for this and possibly related issues, are complete build logs (or failure and success conditions) as attachments
Comment 6 Brent Busby 2017-08-06 21:16:30 UTC 
Created attachment 185104 [details]
python27 build log
Comment 7 Brent Busby 2017-08-06 21:18:35 UTC 
Here's the transcript of the build/install.  There are some errors in there that look readline related, but I don't know how to fix it, because devel/readline is installed as previously suggested, and it happens anyway.
Comment 8 Brent Busby 2017-08-22 14:45:19 UTC 
You can go ahead and close this.  It was caused by accumulated cruft in the basejail of my ezjail installation.  It was solvable by deleting basejail and redoing ezjail-admin -i.
Comment 9 Mark Linimon freebsd_committer freebsd_triage 2017-08-23 10:48:43 UTC 
This was a local problem.
Comment 10 Kubilay Kocak freebsd_committer freebsd_triage 2017-08-25 03:56:42 UTC 
Assogn to committer that resolved