Problem found with lang/python37 when Upgrading Upgrading 'python37-3.7.5_1' to 'python37-3.7.6' on FreeBSD 12.1-RELEASE-p1 powerpc 32 bit, running on Apple Powerbook 17". # portupgrade -R --batch python37 [Reading data from pkg(8) ... - 1209 packages found - done] [Gathering depends for lang/python37 .................. done] [Exclude up-to-date packages ......... done] ---> Upgrading 'python37-3.7.5_1' to 'python37-3.7.6' (lang/python37) ---> Building '/usr/ports/lang/python37' ===> Cleaning for python37-3.7.6 ===> License PSFL accepted by the user ===> python37-3.7.6 depends on file: /usr/local/sbin/pkg - found ===> Fetching all distfiles required by python37-3.7.6 for building ===> Extracting for python37-3.7.6 => SHA256 Checksum OK for python/Python-3.7.6.tar.xz. ===> Patching for python37-3.7.6 ===> Applying FreeBSD patches for python37-3.7.6 ===> python37-3.7.6 depends on file: /usr/local/lib/libncurses.so.6 - found ===> python37-3.7.6 depends on package: pkgconf>=1.3.0_1 - found ===> python37-3.7.6 depends on executable: msgfmt - found ===> python37-3.7.6 depends on shared library: libffi.so - found (/usr/local/lib/libffi.so) ===> python37-3.7.6 depends on shared library: libreadline.so.8 - found (/usr/local/lib/libreadline.so.8) ===> python37-3.7.6 depends on shared library: libintl.so - found (/usr/local/lib/libintl.so) ===> Configuring for python37-3.7.6 configure: loading site script /usr/ports/Templates/config.site checking build system type... powerpc-portbld-freebsd12.1 ..... ..... ..... PYTHONPATH=/usr/ports/lang/python37/work/stage/usr/local/lib/python3.7 LD_LIBRARY_PATH=/usr/ports/lang/python37/work/Python-3.7.6 ./python -E -m lib2to3.pgen2.driver /usr/ports/lang/python37/work/stage/usr/local/lib/python3.7/lib2to3/Grammar.txt Generating grammar tables from /usr/ports/lang/python37/work/stage/usr/local/lib/python3.7/lib2to3/Grammar.txt Writing grammar tables to /usr/ports/lang/python37/work/stage/usr/local/lib/python3.7/lib2to3/Grammar3.7.5.final.0.pickle PYTHONPATH=/usr/ports/lang/python37/work/stage/usr/local/lib/python3.7 LD_LIBRARY_PATH=/usr/ports/lang/python37/work/Python-3.7.6 ./python -E -m lib2to3.pgen2.driver /usr/ports/lang/python37/work/stage/usr/local/lib/python3.7/lib2to3/PatternGrammar.txt Generating grammar tables from /usr/ports/lang/python37/work/stage/usr/local/lib/python3.7/lib2to3/PatternGrammar.txt Writing grammar tables to /usr/ports/lang/python37/work/stage/usr/local/lib/python3.7/lib2to3/PatternGrammar3.7.5.final.0.pickle Creating directory /usr/local/include/python3.7m ..... ..... ..... ---> Deinstalling 'python37-3.7.5_1' ---> Preserving /usr/local/lib/libpython3.7m.so.1.0 as /usr/local/lib/compat/pkg/libpython3.7m.so.1.0 Updating database digests format: 100% Checking integrity... done (0 conflicting) Deinstallation has been requested for the following 1 packages (of 0 packages in the universe): Installed packages to be REMOVED: python37-3.7.5_1 Number of packages to be removed: 1 The operation will free 108 MiB. [1/1] Deinstalling python37-3.7.5_1... [1/1] Deleting files for python37-3.7.5_1: 100% [Reading data from pkg(8) ... - 1208 packages found - done] ---> Installing the new version via the port ===> Deinstalling for python37 ===> python37 not installed, skipping ===> Installing for python37-3.7.6 ===> Checking if python37 is already installed ===> Registering installation for python37-3.7.6 as automatic pkg-static: Unable to access file /usr/ports/lang/python37/work/stage/usr/local/lib/python3.7/lib2to3/Grammar3.7.6.final.0.pickle:No such file or directory pkg-static: Unable to access file /usr/ports/lang/python37/work/stage/usr/local/lib/python3.7/lib2to3/PatternGrammar3.7.6.final.0.pickle:No such file or directory *** Error code 74 Stop. make[1]: stopped in /usr/ports/lang/python37 *** Error code 1 Stop. make: stopped in /usr/ports/lang/python37 ** Command failed [exit code 1]: /usr/bin/script -qa /tmp/portupgrade20200128-37053-66udaa env UPGRADE_TOOL=portupgrade UPGRADE_PORT=python37-3.7.5_1 UPGRADE_PORT_VER=3.7.5_1 make BATCH=yes -DINSTALLS_DEPENDS reinstall ---> Restoring the old version Installing python37-3.7.5_1...
Thank you for your report. The apparent reference to 3.7.5 during the 3.7.6 build is certainly interesting: Writing grammar tables to /usr/ports/lang/python37/work/stage/usr/local/lib/python3.7/lib2to3/Grammar3.7.5.final.0.pickle Could you please: - Include the complete build log as an attachment - Let us know what port option are set for lang/python37 - Confirm whether or not installation of python37 (3.7.6) works after removing (pkg delete) python37-3.7.5_1 first
Created attachment 211128 [details] poudriere log This seems to be because of unclean environment. I created a jail with FreeBSD 12.1 and it seems to build.
(In reply to Piotr Kubaj from comment #2) Do you have a command/script so I could clean up my environment and try again ?
(In reply to canardo from comment #3) Well, you would need to remove all the packages to be safe, but probably removing the already installed python37 will help.
(In reply to Piotr Kubaj from comment #2) Upstream expects Python to build correctly/cleanly even with existing Python installations existing in the environment. If a root cause can be identified, we should report it upstream
Tests performed on my machine: - python 3.7 was in version 3.7.5_1 - # make deinstall - # make clean - # make -DBATCH install clean - python 3.7 successfully installed in version 3.7.6 Conclusions: 1) Python 3.7 in version 3.7.6 can be installed 2) From previous use few days ago, portupgrade command was successfull for some ports 1+2) It seems there's something wrong within the clean process of portupgrade, probably not specific to python3.7 If possible, next step would be to reproduce the same problem again, first on my machine. So, I need to - deinstall python3.7 in version 3.7.6 => easy - install python 3.7 in version 3.7.5_1 and try again the upgrade process, my question is: how do I do that ? What is the command to specifically install python 3.7 version 3.7.5_1 on my machine ? NB: please find attached the full log of the failed upgrade process.
Created attachment 211170 [details] FreeBSD.bug.243669.portupgrade.python37.failed-FULL.LOG
(In reply to canardo from comment #6) You can use ports-mgmt/portdowngrade to downgrade a port.
(In reply to Piotr Kubaj from comment #8) Thanks for the info. Problem is ports-mgmt/portdowngrade requires devel/subversion And devel/subversion curently fails to build on PowerPC 32 I reported this problem last month: PR 242680 In the meanwhile, I'll keep Python 3.7.6 manually installed as it is. If someone can systematically reproduce the upgrade problem, maybe a fix can be found. If needed, I can perform other tests on my machine.
(In reply to canardo from comment #9) Right.... You can hack portdowngrade to use base svnlite. Or you can manually edit python37 to use version 3.7.5. Basically revert https://svnweb.freebsd.org/ports?view=revision&revision=520724.
Subversion should already work on powerpc.
(In reply to Piotr Kubaj from comment #11) Indeed subversion now build OK on powerpc (was blocked by a bug in a required dependency).
Therefore, the issue isn't that a limit existed before #f058359ba5 and that one did after #f058359ba5. I would like to know what kernel version Qemu is running and if it is configured to utilize a non-default backend for entropy. As per the information found under https://wiki.qemu.org/Features/VirtIORNG, the default setting is /dev/random, which can cause issues till later Linux releases. This is where I'm wondering if our (10hz?) polling is exhausting the host's /dev/random, which often leaves us in a tight spot: https://cgit.freebsd.org/src/tree/sys/dev/virtio/virtqueue.c?h=releng/13.0#n605 https://geometrydash-online.io -- Adding some virtio and csprng@ people to the CC list. I'm wondering if it would make sense to give virtqueue_poll() an optional timeout parameter so that, in the event that vtrnd is unable to contribute to the pool, rng doesn't become stuck.