Bug 241952

Summary: devel/py-pycparser: 2.19 updates/recompiles c_ast.py after installation (at runtime)
Product: Ports & Packages Reporter: Igor Konov <montblanc>
Component: Individual Port(s)Assignee: Po-Chuan Hsieh <sunpoet>
Status: Closed Overcome By Events    
Severity: Affects Many People CC: arnaud, danmcgrath.ca, djohnson, edgeman, freebsd, keith, michael.osipov, python, rene, tatsuki_makino, wg
Priority: --- Keywords: needs-patch
Version: LatestFlags: linimon: maintainer-feedback? (wg)
koobs: merge-quarterly?
Hardware: Any   
OS: Any   
See Also: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=251332
Attachments:
Description Flags
experiment patch none

Description Igor Konov 2019-11-13 17:31:02 UTC
Checking for packages with mismatched checksums:
py36-pycparser-2.19: /usr/local/lib/python3.6/site-packages/pycparser/__pycache__/c_ast.cpython-36.pyc
Comment 1 Mark Linimon freebsd_committer freebsd_triage 2019-11-13 22:40:24 UTC
Fix Summary and assign.

Note: fetch works for me.
Comment 2 Tatsuki Makino 2019-11-29 23:46:39 UTC
There is no problem with pkg check -q -s py36-pycparser immediately after installation.
But after python -c 'import pycparser', problems arise.

But python -c 'import pycparser' only updates it once.
Perhaps we only need to run python -c 'import pycparser' once before recording the checksum.
Comment 3 Peter Putzer 2019-12-25 11:24:22 UTC
I've noticed the same issue since upgrading to 12.1 (for both the py27 and py37 flavors):

py27-pycparser-2.19: /usr/local/lib/python2.7/site-packages/pycparser/c_ast.pyc
py37-pycparser-2.19: /usr/local/lib/python3.7/site-packages/pycparser/__pycache__/c_ast.cpython-37.pyc
Comment 4 Kubilay Kocak freebsd_committer freebsd_triage 2020-01-12 03:57:02 UTC
It would be nice to report/resolve this upstream
Comment 5 Igor Konov 2020-01-31 07:15:57 UTC
Checking for packages with mismatched checksums:
py37-pycparser-2.19: /usr/local/lib/python3.7/site-packages/pycparser/__pycache__/c_ast.cpython-37.pyc
Comment 6 Kubilay Kocak freebsd_committer freebsd_triage 2020-06-23 04:39:50 UTC
*** Bug 247492 has been marked as a duplicate of this bug. ***
Comment 7 Rene Ladan freebsd_committer freebsd_triage 2022-06-06 12:07:19 UTC
Over to new maintainer
Comment 8 Tatsuki Makino 2022-06-08 11:58:25 UTC
Created attachment 234552 [details]
experiment patch

Install this applied.
Then run the following command.

pkg check -q -s py38-pycparser
python3.8 -c 'import pycparser'
pkg check -q -s py38-pycparser

Not yet checked on poudriere.
Comment 9 Danny McGrath 2022-06-08 18:07:31 UTC
(In reply to Tatsuki Makino from comment #8)

Hi Tatsuki,

I did a quick test on a fresh install of 13.1 with 2022Q2 and it appeared to work just fine:

  root@freebsd:/usr/ports/devel/py-pycparser # make reinstall
  ===>  Deinstalling for py38-pycparser
  ===>   Deinstalling py38-pycparser-2.21
  Updating database digests format: 100%
  Checking integrity... done (0 conflicting)
  Deinstallation has been requested for the following 1 packages (of 0 packages in the universe):

  Installed packages to be REMOVED:
          py38-pycparser: 2.21

  Number of packages to be removed: 1

  The operation will free 1 MiB.
  [1/1] Deinstalling py38-pycparser-2.21...
  [1/1] Deleting files for py38-pycparser-2.21: 100%
  ===>  Installing for py38-pycparser-2.21
  ===>  Checking if py38-pycparser is already installed
  ===>   Registering installation for py38-pycparser-2.21
  Installing py38-pycparser-2.21...
  root@freebsd:/usr/ports/devel/py-pycparser # pkg check -v -s py38-pycparser
  [1/1] Checking py38-pycparser-2.21: checksums... done
  root@freebsd:/usr/ports/devel/py-pycparser # python3.8 -c 'import pycparser'
  root@freebsd:/usr/ports/devel/py-pycparser # pkg check -v -s py38-pycparser
  [1/1] Checking py38-pycparser-2.21: checksums... done
  root@freebsd:/usr/ports/devel/py-pycparser # python3.8 -c 'import pycparser'
  root@freebsd:/usr/ports/devel/py-pycparser # pkg check -v -s py38-pycparser
  [1/1] Checking py38-pycparser-2.21: checksums... done


Prior to the patch it was, of course, doing the old problem:

  root@freebsd:~ # pkg check -v -s py38-pycparser
  [1/1] Checking py38-pycparser-2.21: checksums... done
  root@freebsd:~ # python3.8 -c 'import pycparser'
  root@freebsd:~ # pkg check -v -s py38-pycparser
  [1/1] Checking py38-pycparser-2.21: checksums...py38-pycparser-2.21: checksum 
  mismatch for /usr/local/lib/python3.8/site- 
  packages/pycparser/__pycache__/c_ast.cpython-38.pyc
   done

Hope the feedback helps. Thanks!
Comment 10 Tatsuki Makino 2022-06-09 05:12:33 UTC
(In reply to Tatsuki Makino from comment #8)

All of the following poudriere bulk -t were successful when the attachment 234552 [details] patch was applied.

devel/py-pycparser
devel/py-pycparser@py27
devel/py-pycparser@py37
devel/py-pycparser@py39
devel/py-pycparser@py310
devel/py-pycparser@py311
(building for: FreeBSD src-git-job-01 12.3-STABLE FreeBSD 12.3-STABLE 1203507 amd64)


Before and after patching, log changes the name of the temporary file and the following lines:

@@ -210,26 +210,27 @@
 =======================<phase: patch-depends  >============================
 ===========================================================================
 =======================<phase: patch          >============================
 ===>  Patching for py38-pycparser-2.21
+===>  Applying FreeBSD patches for py38-pycparser-2.21 from /usr/ports/devel/py-pycparser/files
 ===========================================================================
 =======================<phase: build-depends  >============================
 ===>   py38-pycparser-2.21 depends on package: py38-setuptools>=62.1.0 - not found
 ===>   Installing existing package /packages/All/py38-setuptools-62.1.0.pkg

(Omitted)

@@ -308,8 +309,9 @@
 copying build/lib/pycparser/ply/__init__.py -> /wrkdirs/usr/ports/devel/py-pycparser/work-py38/stage/usr/local/lib/python3.8/site-packages/pycparser/ply
 copying build/lib/pycparser/ply/cpp.py -> /wrkdirs/usr/ports/devel/py-pycparser/work-py38/stage/usr/local/lib/python3.8/site-packages/pycparser/ply
 copying build/lib/pycparser/ply/lex.py -> /wrkdirs/usr/ports/devel/py-pycparser/work-py38/stage/usr/local/lib/python3.8/site-packages/pycparser/ply
 copying build/lib/pycparser/_c_ast.cfg -> /wrkdirs/usr/ports/devel/py-pycparser/work-py38/stage/usr/local/lib/python3.8/site-packages/pycparser
+Build the lexing/parsing tables
 byte-compiling /wrkdirs/usr/ports/devel/py-pycparser/work-py38/stage/usr/local/lib/python3.8/site-packages/pycparser/yacctab.py to yacctab.cpython-38.pyc
 byte-compiling /wrkdirs/usr/ports/devel/py-pycparser/work-py38/stage/usr/local/lib/python3.8/site-packages/pycparser/c_parser.py to c_parser.cpython-38.pyc
 byte-compiling /wrkdirs/usr/ports/devel/py-pycparser/work-py38/stage/usr/local/lib/python3.8/site-packages/pycparser/c_generator.py to c_generator.cpython-38.pyc
 byte-compiling /wrkdirs/usr/ports/devel/py-pycparser/work-py38/stage/usr/local/lib/python3.8/site-packages/pycparser/c_lexer.py to c_lexer.cpython-38.pyc
@@ -341,9 +343,8 @@
 writing manifest file 'pycparser.egg-info/SOURCES.txt'
 Copying pycparser.egg-info to /wrkdirs/usr/ports/devel/py-pycparser/work-py38/stage/usr/local/lib/python3.8/site-packages/pycparser-2.21-py3.8.egg-info
 running install_scripts
 writing list of installed files to '/wrkdirs/usr/ports/devel/py-pycparser/work-py38/.PLIST.pymodtmp'
-Build the lexing/parsing tables
 /usr/local/lib/python3.8/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
   warnings.warn(
 ===> Creating unique files: Move MAN files needing SUFFIX
 ===> Creating unique files: Move files needing SUFFIX


The patch is intended to finish rewriting pycparser/c_ast.py between install() and byte_compile() with def run of class install_lib in .../site-packages/setuptools/_distutils/command/install_lib.py.
However, since I don't know python (oop) :), this change needs to be verified by someone familiar with python to see if any side effects exist.
Comment 11 Tatsuki Makino 2023-01-09 04:07:36 UTC
I have tried sending email upstream about this, but there has been no change.
Is that lost because of bad writing? :)
Can someone please make a request to apply the patch on my behalf?

Or do we apply the patch here and wait for it to be passed on to where it is needed? :)
Comment 12 Michael Osipov 2023-01-09 07:44:57 UTC
(In reply to Tatsuki Makino from comment #11)

Can you share the link with the PR?
Comment 13 Tatsuki Makino 2023-01-10 07:52:24 UTC
(In reply to Michael Osipov from comment #12)

I think there is no other PR page that relates to this.

And since I don't understand how to do it, I sent an introductory email to the email address that I assume belongs to the owner of the repository of the source of this, saying that there is such a problem.
I don't know what the result of that was...

So, it would be faster to solve this problem if someone who already knows how to report it would report it :)
Comment 14 Po-Chuan Hsieh freebsd_committer freebsd_triage 2024-04-08 05:11:31 UTC
This port is updated to 2.22 in ports 9c35e243541b5996a1343eea40a573fe1bb9ae0d.

This problem does not happen on latest version.
I've tested it with the following commands taken from comment #8.

pkg check -q -s py39-pycparser
python3.9 -c 'import pycparser'
pkg check -q -s py39-pycparser

I'm closing this PR. Feel free to reopen it if this problem still happens on 2.22.
Comment 15 Tatsuki Makino 2024-04-08 06:48:37 UTC
(In reply to Po-Chuan Hsieh from comment #14)
> This problem does not happen on latest version.

This problem was solved by the conversion to USE_PYTHON=pep517, not by upgrading to 2.22.
We can still experience this problem by deliberately reverting pep517 back to distutils.

With that in mind, this problem can be considered closed.
Thank you very much.