Bug 209355

Summary: lang/python35: install fails if archivers/lzmalib is installed
Product: Ports & Packages Reporter: Eir Nym <eirnym>
Component: Individual Port(s)Assignee: freebsd-python (Nobody) <python>
Status: Open ---    
Severity: Affects Only Me CC: arved, eirnym, eserte12, koobs, loader, python, w.schwarzenfeld
Priority: --- Keywords: needs-qa
Version: Latest   
Hardware: Any   
OS: Any   
See Also: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=210626
Attachments:
Description Flags
link liblzma.so.5
none
archives/py35-lzma none

Description Eir Nym 2016-05-07 09:51:06 UTC
Python3.[45] fails to install if archivers/lzmalib is installed. The error is _lzma extension can't be found (conflict with Python's plist). If I remove lzmalib from system, _lzma extension will be built and installed.

How to reproduce:

* check if you have no python35 in system
* install *archivers/lzmalib*
* try to install lang/python35

My options for lang/python35 are:
===> The following configuration options are available for python35-3.5.1_2:
     DEBUG=off: Build with debugging support
     IPV6=on: IPv6 protocol support
     LIBFFI=on: Use libffi from ports instead of bundled version
     NLS=on: Enable gettext support for the locale module
     PYMALLOC=on: Enable specialized mallocs
     THREADS=on: Threading support
     TSC=on: Enable processor timestamp counter profiling
====> Hash Algorithm (PEP-456): you can only select none or one of them
     FNV=off: Modified Fowler-Noll-Vo Algorithm
     SIPHASH=on: SipHash24 Algorithm
===> Use 'make config' to modify these settings
Comment 1 Eir Nym 2016-05-08 09:15:04 UTC
It seems to be old bug, but I haven't found it.

I think this is because you can find liblzma in /usr/lib and in /usr/local/lib and the latest doesn't want to be compiled with Python
Comment 2 Tilman Keskinoz freebsd_committer 2016-05-11 16:23:55 UTC
over to python maintainers
Comment 3 Walter Schwarzenfeld freebsd_triage 2016-06-28 19:02:19 UTC
lzmalib is only need by two ports: /archivers/p5-Compress-Raw-Lzma and databases/tokyocabinet. Both have OPTIONS to change.
Comment 4 Eir Nym 2016-08-03 14:05:43 UTC
I want to install and use databases/tokyocabinet with LZMA support and it doesn't accept lzmalib from base. and Python doesn't accept lzmalib from ports. deadlock?
Comment 5 Walter Schwarzenfeld freebsd_triage 2016-08-03 15:23:12 UTC
it seems to work if you add to the lang/python35/Makefile

LIB_DEPENDS=            liblzma.so.1:archivers/lzmalib \
                                    libbz2.so:archivers/bzip2

and create a new pkg-plist with make makeplist or genplist.
I provide no diff, cause I am not sure if the "new" pkg-plist is ok.
Comment 6 Walter Schwarzenfeld freebsd_triage 2016-08-03 15:32:05 UTC
Note:  I have only compiled it with python35. Not really tested. This could only be a workaround for the moment.
Comment 7 Walter Schwarzenfeld freebsd_triage 2016-08-12 01:35:26 UTC
find /usr/ports/lang/python34/work/stage/* -type f|grep lzma
/usr/ports/lang/python34/work/stage/usr/local/lib/python3.4/__pycache__/lzma.cpython-34.pyc
/usr/ports/lang/python34/work/stage/usr/local/lib/python3.4/__pycache__/lzma.cpython-34.pyo
/usr/ports/lang/python34/work/stage/usr/local/lib/python3.4/lib-dynload/_lzma_failed.so
/usr/ports/lang/python34/work/stage/usr/local/lib/python3.4/lzma.py
/usr/ports/lang/python34/work/stage/usr/local/lib/python3.4/test/test_lzma.py
/usr/ports/lang/python34/work/stage/usr/local/lib/python3.4/test/__pycache__/test_lzma.cpython-34.pyc
/usr/ports/lang/python34/work/stage/usr/local/lib/python3.4/test/__pycache__/test_lzma.cpython-34.pyo
----
lib/python3.4/__pycache__/lzma.cpython-34.pyc
lib/python3.4/__pycache__/lzma.cpython-34.pyo
/usr/ports/lang/python34/work/stage/usr/local/lib/python3.4/lib-dynload/_lzma_failed.so
lib/python3.4/lzma.py
lib/python3.4/test/__pycache__/test_lzma.cpython-34.pyc
lib/python3.4/test/__pycache__/test_lzma.cpython-34.pyo
lib/python3.4/test/test_lzma.py
---
the difference is:
/usr/ports/lang/python34/work/stage/usr/local/lib/python3.4/lib-dynload/_lzma_failed.so
and
lib/python3.4/lzma.py


It seems it is only mispelled in pkg-plist. Compare it with python35 pkg-plist (has _lzma_failed.so)
Comment 8 Walter Schwarzenfeld freebsd_triage 2016-08-12 02:07:01 UTC
correct a typo:
the difference is:
stage:
/usr/ports/lang/python34/work/stage/usr/local/lib/python3.4/lib-dynload/_lzma_failed.so
and
pkg-plist
lib/python3.4/_lzma.py
Comment 9 Fukang Chen freebsd_committer 2016-09-04 16:08:37 UTC
It looks it tries to link the /usr/local/lib/liblzma.so instead of /usr/lib/liblzma.so.
and liblzma.so from the port archives/lzmalib doesn't have "lzma_properties_size".

The first "-L/usr/local/lib" in the "cc -shared" comes from Mk/readline.mk, 
it make the linker to search this directory before the default directories.

-------------------------------------------------------------
building '_lzma' extension
cc -fPIC -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -O -pipe -fno-strict-aliasing -Werror=declaration-after-statement -I./Include -I. -IInclude -I/usr/local/include -I/usr/ports/lang/python35/work/Python-3.5.2/Include -I/usr/ports/lang/python35/work/Python-3.5.2 -c /usr/ports/lang/python35/work/Python-3.5.2/Modules/_lzmamodule.c -o build/temp.freebsd-12.0-CURRENT-arm-3.5/usr/ports/lang/python35/work/Python-3.5.2/Modules/_lzmamodule.o
cc -shared -lpthread -Wl,-rpath=/usr/lib:/usr/local/lib -L/usr/local/lib -lpthread -Wl,-rpath=/usr/lib:/usr/local/lib -L/usr/local/lib -O -pipe -fno-strict-aliasing -I/usr/local/include -I/usr/local/include build/temp.freebsd-12.0-CURRENT-arm-3.5/usr/ports/lang/python35/work/Python-3.5.2/Modules/_lzmamodule.o -L. -L/usr/local/lib -llzma -lpython3.5m -o build/lib.freebsd-12.0-CURRENT-arm-3.5/_lzma.so
*** WARNING: renaming "_lzma" since importing it failed: build/lib.freebsd-12.0-CURRENT-arm-3.5/_lzma.so: Undefined symbol "lzma_properties_size"
Comment 10 Fukang Chen freebsd_committer 2016-09-04 16:18:11 UTC
Created attachment 174374 [details]
link liblzma.so.5

workaround 1

The port archives/lzmalibs provides /usr/local/lib/liblzma.so.1
and the base has /usr/lib/liblzma.so.5. 

If there exists a /usr/local/lib/liblzma.so.1. then pass the "-l:liblzma.so.5"
to make it link the library in the base.
Comment 11 Fukang Chen freebsd_committer 2016-09-04 16:41:43 UTC
Created attachment 174375 [details]
archives/py35-lzma

workaround 2

disable compiling _lzma in lang/python35 and move it to a separate port.
there would be no USES= readline in that port and no version number of 
the base liblzma.so in the patch.

Just my thoughts, not proposing these workarounds as the fix.
Comment 12 Fukang Chen freebsd_committer 2016-09-04 16:49:36 UTC
(In reply to Fukang Chen from comment #9)

s|Mk/readlink.mk|Mk/Uses/readline.mk|
Comment 13 eserte12 2017-02-20 07:02:58 UTC
See also https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=217244
Comment 14 Walter Schwarzenfeld freebsd_triage 2018-01-19 00:22:19 UTC
Any news here?
Comment 15 Rene Ladan freebsd_committer 2018-12-18 09:38:11 UTC
removed expired python34 from subject.