Bug 203021 - lang/python{27,33,34,35,36}: Should install GDB integration tool(s)
Summary: lang/python{27,33,34,35,36}: Should install GDB integration tool(s)
Status: Closed FIXED
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Some People
Assignee: Kubilay Kocak
URL: https://reviews.freebsd.org/D10398
Keywords: feature, needs-qa
Depends on:
Blocks:
 
Reported: 2015-09-10 19:05 UTC by Conrad Meyer
Modified: 2018-01-24 23:16 UTC (History)
5 users (show)

See Also:
koobs: maintainer-feedback+


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Conrad Meyer freebsd_committer freebsd_triage 2015-09-10 19:05:41 UTC
It's in Tools/gdb/libpython.py in Python 2.7+.
Comment 1 Conrad Meyer freebsd_committer freebsd_triage 2017-04-13 22:25:38 UTC
It appears Python port is abandoned.
Comment 2 Mathieu Arnold freebsd_committer freebsd_triage 2017-04-13 23:20:00 UTC
Over to maintainer.
Comment 3 Mathieu Arnold freebsd_committer freebsd_triage 2017-04-13 23:20:35 UTC
Note, this is a volunteer project, if you really want something, might be a good idea to provide a patch.
Comment 4 Conrad Meyer freebsd_committer freebsd_triage 2017-04-13 23:25:07 UTC
I'm a volunteer too, Mathieu.
Comment 5 Conrad Meyer freebsd_committer freebsd_triage 2017-04-13 23:39:38 UTC
E.g.,

Index: Makefile
===================================================================
--- Makefile    (revision 438066)
+++ Makefile    (working copy)
@@ -125,6 +125,8 @@
 post-install:
        for i in ${STAGEDIR}${PREFIX}/lib/python2.7/lib-dynload/*.so; do \
                ${STRIP_CMD} $$i; done                                          # Strip shared extensions
+       install -m444 ${WRKSRC}/Tools/gdb/libpython.py \
+               ${STAGEDIR}${PREFIX}/debug/${LOCALBASE}/lib/libpython2.7.so.1.0.debug-gdb.py

 .include <bsd.port.pre.mk>

Index: pkg-plist
===================================================================
--- pkg-plist   (revision 438066)
+++ pkg-plist   (working copy)
@@ -4545,3 +4545,4 @@
 lib/python2.7/zipfile.pyo
 libdata/pkgconfig/python-2.7.pc
 man/man1/python2.7.1.gz
+debug/${LOCALBASE}/lib/libpython2.7.so.1.0.debug-gdb.py
Comment 6 Conrad Meyer freebsd_committer freebsd_triage 2017-04-14 00:27:46 UTC
The path "${LOCALBASE}/lib/libpython2.7.so.1-gdb.py" may also work.  That's what we use at $DAYJOB, although we may modify the default GDB search path to make that work.
Comment 7 Kubilay Kocak freebsd_committer freebsd_triage 2017-04-14 12:38:40 UTC
@Conrad 

Could you include your patch as an attachment please, or alternatively (preferred), open a Phabricator review (adding the review URl to the URl field in bugzilla).

Can you also confirm:

- That the patch passes QA (poudriere)
- To what extent this change should be made across all Python port versions. Ideally, include the changes for the other port versions in the patch/review as well
- Whether this change (the addition of the file) should only be made when the DEBUG port option is on, or unconditionally, with rationale if possible.
Comment 8 Conrad Meyer freebsd_committer freebsd_triage 2017-04-14 16:30:39 UTC
(In reply to Kubilay Kocak from comment #7)
> Could you include your patch as an attachment please, or alternatively (preferred), open a Phabricator review (adding the review URl to the URl field in bugzilla).

Done.

> Can you also confirm:
> 
> - That the patch passes QA (poudriere)

Do we have any cluster machines that can be used for poudriere?  I'm moving in two days and my FreeBSD machines are packed.

> - To what extent this change should be made across all Python port versions. Ideally, include the changes for the other port versions in the patch/review as well

It should be done for all Python versions (and the patch does).

> - Whether this change (the addition of the file) should only be made when the DEBUG port option is on, or unconditionally, with rationale if possible.

I don't see any reason to limit it to DEBUG.
Comment 9 Conrad Meyer freebsd_committer freebsd_triage 2017-04-28 12:23:42 UTC
The patch is here if anyone will commit it: https://reviews.freebsd.org/D10398

Unfortunately, no one from python@ will review it and I am not allowed to commit it.
Comment 10 Kubilay Kocak freebsd_committer freebsd_triage 2017-05-26 08:05:49 UTC
@Conrad, during QA (with DEVELOPER=yes in /etc/make.conf), I observe the following:

====> Running Q/A tests (stage-qa)
readelf: Not an ELF file.
Warning: /wrkdirs/usr/ports/lang/python27/work/stage/usr/local/lib/libpython2.7.so.1-gdb.py doesn't have a SONAME.
Warning: pkg(8) will not register it as being provided by the port.
Warning: If another port depend on it, pkg will not be able to know where it comes from.
Warning: It is directly in /usr/local/lib, it is probably used by other ports.

I'm not sure what this might mean for desired functionality out of the box for the port or binary package. This may not be an issue if no other ports depend (LIB_DEPENDS) on it.

Have you tested whether GDB works as expected with libpython2.7.so.1-gdb.py in that location and confirmed it works without additional intervention once installed?

CC'ing Baptiste who might be able to provide some guidance.
Comment 11 Kubilay Kocak freebsd_committer freebsd_triage 2017-05-26 08:08:14 UTC
It just occurred to me that this may just be a *.so file name matching thing. 

If so, is it possible (from a 'it works out of the box') point of view to either rename the file and/or put it in a still-suitable location not inside LOCALBASE/lib?
Comment 12 Conrad Meyer freebsd_committer freebsd_triage 2017-05-26 14:17:19 UTC
(In reply to Kubilay Kocak from comment #10)
> @Conrad, during QA (with DEVELOPER=yes in /etc/make.conf), I observe the following:
> 
> ====> Running Q/A tests (stage-qa)
> readelf: Not an ELF file.
> Warning: /wrkdirs/usr/ports/lang/python27/work/stage/usr/local/lib/libpython2.7.so.1-gdb.py
> doesn't have a SONAME.
> Warning: pkg(8) will not register it as being provided by the port.
> Warning: If another port depend on it, pkg will not be able to know where it comes from.
> Warning: It is directly in /usr/local/lib, it is probably used by other ports.

This is a spurious warning -- it isn't a shared object and obviously nothing can use it like one.

> I'm not sure what this might mean for desired functionality out of the box for the port
> or binary package. This may not be an issue if no other ports depend (LIB_DEPENDS) on it.

I don't think it's an issue -- it's not a shared object, nothing can LIB_DEPENDS on it.

> Have you tested whether GDB works as expected with libpython2.7.so.1-gdb.py in that
> location and confirmed it works without additional intervention once installed?

Yep, see comment #6.  We use it pretty frequently at work.

# ls /usr/local/lib/libpython2.7.so.1*
/usr/local/lib/libpython2.7.so.1              /usr/local/lib/libpython2.7.so.1-gdb.py
# echo -e "import os\nos.abort()" > a.py
# gdb771 --args python a.py
...
(gdb) r
...
Program received signal SIGABRT, Aborted.
[Switching to Thread 8006c1400 (LWP 100708)]
0x000000080129d84a in thr_kill () from /lib/libc.so.7
(gdb) py-bt
Traceback (most recent call first):
  File "a.py", line 2, in <module>
    os.abort()
(gdb) py-list
   1    import os
  >2    os.abort()


(In reply to Kubilay Kocak from comment #11)
> It just occurred to me that this may just be a *.so file name matching thing. 

Yep.

> If so, is it possible (from a 'it works out of the box') point of view to either rename
> the file and/or put it in a still-suitable location not inside LOCALBASE/lib?

The name can't be changed, I think.  There are a few suitable locations.  Fedora uses a location associated with the separated shared object debuginfo, which ports doesn't support:

/usr/lib/debug/usr/lib64/libpython2.7.so.1.0.debug-gdb.py

Note that this portion   ^^^^^^^^^^^^^^^^^^^^^^^^^  has to match some file GDB is loading already.  I'm not sure what search paths GDB has by default.  In the differential review, someone mentioned:

/usr/local/share/gdb/auto-load/usr/local/lib/libglib-2.0.so.0.5000.2-gdb.py

This is discussed a little bit more in the first few comments of the review.
Comment 13 commit-hook freebsd_committer freebsd_triage 2017-06-06 12:57:46 UTC
A commit references this bug:

Author: koobs
Date: Tue Jun  6 12:56:49 UTC 2017
New revision: 442759
URL: https://svnweb.freebsd.org/changeset/ports/442759

Log:
  lang/python{27,33,34,35,36}: Install GDB debugging script

  Users with a GDB that supports [1] Python extensions will automatically
  load the extra debugging extensions when debugging programs that are
  linked with libpythonX.Y.so.foo.

  This enables extensions like 'py-bt' and 'py-frame' as described in
  the Fedora Wiki Article: Easier Python Debugging [2], which can be
  useful for debugging Python program state from crashes in C extensions,
  for example.

  [1] PYTHON option enabled in devel/gdb
  [2] https://fedoraproject.org/wiki/Features/EasierPythonDebugging

  PR:		203021
  Submitted by:	cem
  Reviewed by:	mat, koobs (python)
  Approved by:	koobs (python)
  Differential Revision: D10398

Changes:
  head/lang/python27/Makefile
  head/lang/python27/pkg-plist
  head/lang/python33/Makefile
  head/lang/python33/pkg-plist
  head/lang/python34/Makefile
  head/lang/python34/pkg-plist
  head/lang/python35/Makefile
  head/lang/python35/pkg-plist
  head/lang/python36/Makefile
  head/lang/python36/pkg-plist
Comment 14 Kubilay Kocak freebsd_committer freebsd_triage 2017-06-06 13:02:24 UTC
Thank you for the feature addition to Python/GDB Conrad, apologies it took so long to get it in.
Comment 15 Conrad Meyer freebsd_committer freebsd_triage 2017-06-06 14:46:59 UTC
Thanks for getting it in, Kubilay.
Comment 16 Kubilay Kocak freebsd_committer freebsd_triage 2017-06-06 14:49:14 UTC
(In reply to Conrad Meyer from comment #15)

My pleasure, you're welcome
Comment 17 Mikhail T. 2017-06-07 14:03:04 UTC
I tried this after rebuilding lang/python36 and devel/gdb with the following options:

_OPTIONS_READ=python36-3.6.1_1
_FILE_COMPLETE_OPTIONS_LIST=DEBUG IPV6 LIBFFI NLS PYMALLOC THREADS FNV SIPHASH
OPTIONS_FILE_UNSET+=DEBUG
OPTIONS_FILE_UNSET+=IPV6
OPTIONS_FILE_SET+=LIBFFI
OPTIONS_FILE_SET+=NLS
OPTIONS_FILE_SET+=PYMALLOC
OPTIONS_FILE_SET+=THREADS
OPTIONS_FILE_UNSET+=FNV
OPTIONS_FILE_SET+=SIPHASH

and:

_OPTIONS_READ=gdb-7.12.1_1
_FILE_COMPLETE_OPTIONS_LIST=DEBUG GDB_LINK GUILE KGDB PYTHON TUI BUNDLED_READLINE PORT_READLINE
OPTIONS_FILE_UNSET+=DEBUG
OPTIONS_FILE_SET+=GDB_LINK
OPTIONS_FILE_UNSET+=GUILE
OPTIONS_FILE_SET+=KGDB
OPTIONS_FILE_SET+=PYTHON
OPTIONS_FILE_SET+=TUI
OPTIONS_FILE_UNSET+=BUNDLED_READLINE
OPTIONS_FILE_SET+=PORT_READLINE

Attempting `py-bt' gave me the following error:

(gdb) py-bt
Python Exception <type 'exceptions.ValueError'> Variable 'func_obj' not found.: 
Error occurred in Python command: Variable 'func_obj' not found.

I then found this Python-ticket: https://bugs.python.org/issue30345 and tried the patch referenced there (https://github.com/python/cpython/commit/d05f7fdf6cf77724bd3064fb5a0846ef5cfe0c88). Unfortunately, that didn't help:

(gdb) py-frame
Undefined command: "py-frame".  Try "help".
(gdb) py-bt
Python Exception <type 'exceptions.ValueError'> Variable 'func_obj' not found.: 
Error occurred in Python command: Variable 'func_obj' not found.

Looks like something is still missing -- at least, in the case of 3.6.
Comment 18 Kubilay Kocak freebsd_committer freebsd_triage 2017-06-07 23:49:59 UTC
Thank you for the report Mikhail.

In future for these kinds of issues, please create a new issue referencing (see also) the original. 

That is to say: issues landing a feature, and bugs/issues identified with that feature after release should be separate (but linked/referenced). 

This ensures titles, investigations and conversation, and assignees can remain distinct and accurate.
Comment 19 Kubilay Kocak freebsd_committer freebsd_triage 2017-06-07 23:50:24 UTC
Request feedback from Conrad
Comment 20 Conrad Meyer freebsd_committer freebsd_triage 2017-06-07 23:58:53 UTC
I also think this bug should be followed up in a separate issue.