Created attachment 228348 [details] xz-compressed logs of pkg -d upgrade en-freebsd-doc, three subsequent runs starting from a prior installed en-freebsd-doc-20210125,1 Greetings, several of my machines require two "pkg upgrade" runs to upgrade the en-freebsd-doc and de-freebsd-doc packages from 20210125,1 to 20210918,1. There is no note of this in ports/CHANGES. The relevant change is apparently 7ee8c0803bbf5763c4c84bf23ccef422db9a692b. The workaround appears to be pkg delete-ing the old *-freebsd-doc packages and reinstalling them by name afterwards. Please add a note of this to ports/CHANGES. Data Points as Debugging Aids: * I have rebuilt the old package from ports 84ebfb568a682d804cdcf77514670c5085a4f368 for FreeBSD:13:* - get it from https://people.freebsd.org/~mandree/en-freebsd-doc-20210125,1.pkg (35 MibiBytes) * 1st run: pkg: Fail to rename /usr/local/share/doc/freebsd/en/articles/bsdl-gpl/.pkgtemp.article.html.bYSz3D04nZod -> /usr/local/share/doc/freebsd/en/articles/bsdl-gpl/article.html:No such file or directory * 2nd run: pkg: unlinkat(usr/local/share/doc/freebsd/en): Operation not permitted Relevant contents (sorry, de_DE locale for dates but you get the file types): Old package en-freebsd-doc: 20210125,1 $ ls -ldb /usr/local/share/doc/freebsd/en lrwxr-xr-x 1 root wheel 15 1 Okt. 09:27 /usr/local/share/doc/freebsd/en -> en_US.ISO8859-1 New package en-freebsd-doc: 20210918,1 $ ls -ldb /usr/local/share/doc/freebsd/en drwxr-xr-x 7 root wheel 7 1 Okt. 09:41 en NOTE: Failed upgrades appear to leave the old en_US.ISO8859-1 and other directories behind, and also .pkg* temp files. Longer logs: see attachment for pkg -d upgrade logs. Reference screenshots of plain pkg upgrade: First pkg run aborts: ... Checking for upgrades (7 candidates): 100% Processing candidates (7 candidates): 100% The following 2 package(s) will be affected (of 0 checked): Installed packages to be UPGRADED: automake: 1.16.3 -> 1.16.4 [FreeBSD-latest] en-freebsd-doc: 20210125,1 -> 20210918,1 [FreeBSD-latest] Number of packages to be upgraded: 2 The operation will free 13 MiB. 20 MiB to be downloaded. Proceed with this action? [y/N]: y [1/2] Fetching en-freebsd-doc-20210918,1.pkg: 100% 20 MiB 1.9MB/s 00:11 [2/2] Fetching automake-1.16.4.pkg: 100% 550 KiB 562.9kB/s 00:01 Checking integrity... done (0 conflicting) [1/2] Upgrading en-freebsd-doc from 20210125,1 to 20210918,1... [1/2] Extracting en-freebsd-doc-20210918,1: 100% pkg: Fail to rename /usr/local/share/doc/freebsd/en/articles/bsdl-gpl/.pkgtemp.article.html.bYSz3D04nZod -> /usr/local/share/doc/freebsd/en/articles/bsdl-gpl/article.html:No such file or directory Second run emits a truckload of errors (see log) but succeeds: [root@rsv1 ~]# pkg upgrade -y Updating FreeBSD-latest repository catalogue... FreeBSD-latest repository is up to date. Updating local repository catalogue... local repository is up to date. All repositories are up to date. Checking for upgrades (7 candidates): 100% Processing candidates (7 candidates): 100% Checking integrity... done (0 conflicting) The following 2 package(s) will be affected (of 0 checked): Installed packages to be UPGRADED: automake: 1.16.3 -> 1.16.4 [FreeBSD-latest] en-freebsd-doc: 20210125,1 -> 20210918,1 [FreeBSD-latest] Number of packages to be upgraded: 2 The operation will free 13 MiB. [1/2] Upgrading en-freebsd-doc from 20210125,1 to 20210918,1... [1/2] Extracting en-freebsd-doc-20210918,1: 100% pkg: unlinkat(usr/local/share/doc/freebsd/en): Operation not permitted en-freebsd-doc-20210125,1: missing file /usr/local/share/doc/freebsd/en_US.ISO8859-1/articles/bsdl-gpl/addenda.html en-freebsd-doc-20210125,1: missing file /usr/local/share/doc/freebsd/en_US.ISO8859-1/articles/bsdl-gpl/article.html ... en-freebsd-doc-20210125,1: missing file /usr/local/share/doc/freebsd/en_US.ISO8859-1/books/porters-handbook/versions.html en-freebsd-doc-20210125,1: missing file /usr/local/share/doc/freebsd/en_US.ISO8859-1/books/porters-handbook/why-port.html en-freebsd-doc-20210125,1: missing file /usr/local/share/doc/freebsd/faq en-freebsd-doc-20210125,1: missing file /usr/local/share/doc/freebsd/handbook en-freebsd-doc-20210125,1: missing file /usr/local/share/licenses/en-freebsd-doc-20210125,1/BSD2CLAUSE en-freebsd-doc-20210125,1: missing file /usr/local/share/licenses/en-freebsd-doc-20210125,1/LICENSE en-freebsd-doc-20210125,1: missing file /usr/local/share/licenses/en-freebsd-doc-20210125,1/catalog.mk [2/2] Upgrading automake from 1.16.3 to 1.16.4... [2/2] Extracting automake-1.16.4: 100%
Note that the first, failing pkg upgrade run actually crashes after logging the error given above.
Hi, Thanks for this feedback. About CHANGES, I had the feeling that UPDATING should be more correct (I may be wrong)? Is something like the following correct?: 20210927: AFFECTS: user of misc/freebsd-doc-* AUTHOR: blackend@FreeBSD.org The misc/freebsd-doc-* ports have been updated to the new doc toolchain. Some problems may occur during an upgrade. Users are encouraged to delete the misc/freebsd-doc-* ports, then reinstall the new ones without using the pkg upgrade facility.
Oh of course it's UPDATING not CHANGES which I had meant to propose (but did not write). But this is merely a workaround, and we should fix pkg instead to handle such upgrades properly.
(In reply to Marc Fonvieille from comment #2) The text itself seems fine for me.
On one of my machines where pkg aborted, after deinstallation of en-freebsd-doc, I have a truckload of .pkgtemp* garbage files and leftover directories lying around, find prints 458 lines: # find /usr/local/share/doc/freebsd/ /usr/local/share/doc/freebsd/ /usr/local/share/doc/freebsd/en_US.ISO8859-1 /usr/local/share/doc/freebsd/en_US.ISO8859-1/articles /usr/local/share/doc/freebsd/en_US.ISO8859-1/articles/bsdl-gpl /usr/local/share/doc/freebsd/en_US.ISO8859-1/articles/bsdl-gpl/.pkgtemp.article.html.bYSz3D04nZod /usr/local/share/doc/freebsd/en_US.ISO8859-1/articles/bsdl-gpl/.pkgtemp.article.pdf.V0SSyVb0QZzJ /usr/local/share/doc/freebsd/en_US.ISO8859-1/articles/building-products /usr/local/share/doc/freebsd/en_US.ISO8859-1/articles/building-products/.pkgtemp.article.html.tFnRvYuSyvAa /usr/local/share/doc/freebsd/en_US.ISO8859-1/articles/building-products/.pkgtemp.article.pdf.Gp96ab4pxlSz ...
Another proposal: 20210927: AFFECTS: user of misc/freebsd-doc-* AUTHOR: blackend@FreeBSD.org The misc/freebsd-doc-* ports have been updated to the new doc toolchain, and use a somewhat different arrangement of directories and symlinks, which pkg 1.17 does not handle well. Users are encouraged to delete the misc/freebsd-doc-* ports, then reinstall the new ones without using the pkg upgrade facility.
I just found misc/freebsd-doc-all wasn't in sync with others misc/freebsd-doc-* (see https://cgit.freebsd.org/ports/commit/?id=a10f21cbae209a6fbafcdf4d60c2c1e125c137ec). Could you check if it solves the issue for you? I think the problem is elsewhere: between both versions of the ports, some plists and files have been removed. Some install sub-directories have been changed. These changes could explain why pkg has issue with upgrade. I did not commit the UPDATING entry yet, I'm waiting for your feedback for the change on misc/freebsd-doc-all.
For sure this is a pkg bug, the issue being I think: doc/en becomes a directory and pkg fails at handling it correctly during upgrade. When such things happen pkg should detect it, drop its "as atomic as possible way to upgrade" and find a new upgrade strategy. This will take time to fix correctly but at least it is reproducible!!
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/ports/commit/?id=c1e0478c8f5a12efda1abcae69612849853d7cdd commit c1e0478c8f5a12efda1abcae69612849853d7cdd Author: Marc Fonvieille <blackend@FreeBSD.org> AuthorDate: 2021-10-06 17:41:28 +0000 Commit: Marc Fonvieille <blackend@FreeBSD.org> CommitDate: 2021-10-06 17:41:28 +0000 UPDATING: Document misc/freebsd-doc-* upgrade misc/freebsd-doc-* need to be deleted by end. pkg fails at handling correctly new directories. PR: 258858 Reported by: Matthias Andree (mandree) UPDATING | 11 +++++++++++ 1 file changed, 11 insertions(+)
(In reply to Baptiste Daroussin from comment #8) Ok. I updated UPDATING. Do I need to let this PR open as a reminder for pkg bug?
^Triage: committed back in 2021.
Just noting that this appears to remain unfixed. I hit this with pkg-1.20.9_1: [1/1] Upgrading en-freebsd-doc from 53120,1 to 20240216,1... [1/1] Extracting en-freebsd-doc-20240216,1: 100% pkg: Fail to rename /usr/local/share/doc/freebsd/en/articles/.pkgtemp.freebsd-src-lsp.oNnAEcf17jhi -> /usr/local/share/doc/freebsd/en/articles/freebsd-src-lsp:No such file or directory and so on as per Matthias' description.