Bug 251332 - sysutils/py-salt: salt_master modifies bytecode of py-pycparser (c_ast.cpython-37.pyc)
Summary: sysutils/py-salt: salt_master modifies bytecode of py-pycparser (c_ast.cpytho...
Status: Open
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Many People
Assignee: Kirill Ponomarev
URL:
Keywords: needs-qa
Depends on:
Blocks:
 
Reported: 2020-11-23 16:48 UTC by Danny McGrath
Modified: 2022-10-01 12:49 UTC (History)
7 users (show)

See Also:
koobs: maintainer-feedback? (krion)


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Danny McGrath 2020-11-23 16:48:08 UTC
For months now I have been getting the following checksum error during a `pkg check -sv py37-pycparser-2.20` scan:

Checking py37-pycparser-2.20: checksums...py37-pycparser-2.20: checksum mismatch for /usr/local/lib/python3.7/site-packages/pycparser/__pycache__/c_ast.cpython-37.pyc

Is there any way to fix this? A reinstall doesn't seem to help. I recall someone on irc mentioning it was affecting them as well, but I can't recall the details.

Thanks!
Comment 1 Po-Chuan Hsieh freebsd_committer freebsd_triage 2020-11-24 19:12:58 UTC
I don't think it's devel/py-pycparser's problem.
I have no idea what would cause this.
It seems that the bytecode is modified after devel/py-pycparser installation.
Does it happen right after you build and install devel/py-pycparser?
If not, I would guess it is caused by something run with root permission.
Please check the timestamp of c_ast.cpython-37.pyc and find the suspect.
Comment 2 Danny McGrath 2020-11-24 19:40:37 UTC
In terms of reproducing the problem, it is saltstack (py37-salt) minion restarts that cause the file to regenerate.

I did some digging and seems I have indeed reported this once and it was closed as a dupe:

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=247492

From irc, koobs was originally telling me that:

"the setup.py needs to be modified to include compiling that file during build, so the file is listed in install --record output"

Perhaps this helps!
Comment 3 Po-Chuan Hsieh freebsd_committer freebsd_triage 2020-11-24 19:51:35 UTC
(In reply to Danny McGrath from comment #2)

> I did some digging and seems I have indeed reported this once and it was closed as a dupe:
> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=247492

OK, I wasn't the maintainer then.

> From irc, koobs was originally telling me that:
> "the setup.py needs to be modified to include compiling that file during build, so the file is listed in install --record output"

No, that's not the problem. "install --record output" tells the packaging list. The .pyc was already packaged (with checksum), otherwise you won't see it begin checked by pkg.
Comment 4 Po-Chuan Hsieh freebsd_committer freebsd_triage 2020-11-24 19:55:52 UTC
% pkg info -lx pycparser | grep c_ast
        /usr/local/lib/python3.9/site-packages/pycparser/__pycache__/c_ast.cpython-39.opt-1.pyc
        /usr/local/lib/python3.9/site-packages/pycparser/__pycache__/c_ast.cpython-39.pyc
        /usr/local/lib/python3.9/site-packages/pycparser/_c_ast.cfg
        /usr/local/lib/python3.9/site-packages/pycparser/c_ast.py
% sudo pkg check -svx pycparser
[1/1] Checking py39-pycparser-2.20: checksums... done
Comment 5 Danny McGrath 2020-11-24 20:01:27 UTC
Here are the steps that I use:

<pts/3|root|saltmaster|~ #> service salt_minion stop
Stopping salt_minion.
Waiting for PIDS: 91900.
<pts/3|root|saltmaster|~ #> service salt_master stop
Stopping salt_master.
Waiting for PIDS: 30339.
<pts/3|root|saltmaster|~ #> pkg check -svx pycparser
[1/1] Checking py37-pycparser-2.20: checksums...py37-pycparser-2.20: checksum mismatch for /usr/local/lib/python3.7/site-packages/pycparser/__pycache__/c_ast.cpython-37.pyc
 done
<pts/3|root|saltmaster|~ #> pkg upgrade -f py37-pycparser
Updating poudriere repository catalogue...
poudriere repository is up to date.
All repositories are up to date.
The following 1 package(s) will be affected (of 0 checked):

Installed packages to be REINSTALLED:
        py37-pycparser-2.20

Number of packages to be reinstalled: 1

164 KiB to be downloaded.

Proceed with this action? [y/N]: y
[1/1] Fetching py37-pycparser-2.20.txz: 100%  164 KiB 168.4kB/s    00:01    
Checking integrity... done (0 conflicting)
[1/1] Reinstalling py37-pycparser-2.20...
[1/1] Extracting py37-pycparser-2.20: 100%
<pts/3|root|saltmaster|~ #> pkg check -svx pycparser
[1/1] Checking py37-pycparser-2.20: checksums... done
<pts/3|root|saltmaster|~ #> service salt_master start
Starting salt_master.
<pts/3|root|saltmaster|~ #> pkg check -svx pycparser
[1/1] Checking py37-pycparser-2.20: checksums...py37-pycparser-2.20: checksum mismatch for /usr/local/lib/python3.7/site-packages/pycparser/__pycache__/c_ast.cpython-37.pyc
 done
Comment 6 Po-Chuan Hsieh freebsd_committer freebsd_triage 2020-11-24 20:11:08 UTC
Then you'll need to consult py-salt maintainer why it regenerates the python bytecode.

Add him to Cc list.
Comment 7 Danny McGrath 2021-11-18 16:10:46 UTC
Any word on this bug report? It still appears to be a problem in 2021Q4 with 13.0.

Thank you.
Comment 8 Kubilay Kocak freebsd_committer freebsd_triage 2022-02-07 06:42:09 UTC
Modulo salt *actually* modifying pycparsers (.py) code, deleting the original pyc such that its re-generated or modifying its timestamp, this would appear to be a duplicate of bug 241952 (the original for duped bug 247492 as referenced in comment 3) 

As bug 241952 states: "But after python -c 'import pycparser', problems arise.", so pycparser has the regeneration issue on initial install, independently of salt, or any other triggers.
Comment 9 Kubilay Kocak freebsd_committer freebsd_triage 2022-02-07 06:47:39 UTC
We need positive evidence, or details of the exact nature of how salt is apparently triggering the behaviour, in order to warrant not treating this issue as a dupe of 241952.

An example of that would be that the timestamp of pycparser files changes each and every time salt_master is run. If it remains the same, then this leaves open the assessment that it's a first-run only [1][2] (of pycparser) issue.

[1] This would occur at first-run on every pycparser port/package update.
[2] If pkg leaves behind the regenerated .pyc due to checksum mismatch the behaviour may be different
Comment 10 Tatsuki Makino 2022-06-05 10:44:36 UTC
def dependency_information of ${WRKSRC}/salt/version.py has __import__.
If this __import__ means dynamically importing other modules, then the code around here is probably importing pycparser.

...maybe :)