Summary: | Portsnap continuously fails with "snapshot corrupt" | ||
---|---|---|---|
Product: | Services | Reporter: | Patrick McMunn <doctorwhoguy> |
Component: | Portsnap | Assignee: | Colin Percival <cperciva> |
Status: | Closed FIXED | ||
Severity: | Affects Only Me | CC: | chris, joneum, lwhsu, pi |
Priority: | --- | ||
Version: | unspecified | ||
Hardware: | Any | ||
OS: | Any |
Description
Patrick McMunn
2020-04-30 19:54:53 UTC
Apparently the tree was updated and the latest snapshot correct because today I deleted the portsnap database and the ports tree and ran "portsnap auto" and got a ports tree with no errors. This seems to indicate there was an upstream issue that has since been resolved (at least for this particular incident). But if it is an upstream issue, I am still concerned because of how frequently it has been occurring over the past few months. How often did it occur in the last few month ? And: can you check/report which portsnap site is used ? My poudriere reports where I get it: [00:00:00] Updating portstree "default" with portsnap...Looking up portsnap.FreeBSD.org mirrors... 6 mirrors found. Fetching snapshot tag from ec2-eu-west-1.portsnap.freebsd.org... done. Sorry, found your report detail. Ignore my last comment 8-} Feedback timeout. Please feel free to reopen this if you still see the same issue. Reopening. I apologize for not following up in a timely manner. I must have missed the email notifications that there were any responses. The issue also cleared up for a while until a day or two ago when it occurred again. So I deleted my ports tree and Portsnap database then ran "portsnap auto". It downloaded and extracted just fine. But then I tried to update the tree this afternoon, and it happened again. The thing that baffles me is the errors about "no such file or directory" for /dev/stdout even though I can clearly see the device when I run ls. Is it possible that the snapshots themselves are fine but there is some error happening locally that causes them to not work? Of course, that's just pure conjecture on my part. Below is the output from this afternoon. root@dl380g7:/usr/ports # portsnap auto Looking up portsnap.FreeBSD.org mirrors... 4 mirrors found. Fetching snapshot tag from ipv4.aws.portsnap.freebsd.org... done. Fetching snapshot metadata... done. Updating from Sat Jul 18 13:07:10 CDT 2020 to Mon Jul 20 17:11:43 CDT 2020. Fetching 4 metadata patches./usr/sbin/portsnap: cannot create /dev/stdout: No such file or directory done. Applying metadata patches... done. cut: /dev/stdin: No such file or directory Fetching 4 metadata files... done. Fetching 173 patches. /usr/sbin/portsnap: cannot create /dev/stdout: No such file or directory (0/173) 0.00% done. done. Applying patches... done.pping 72bb2fb5de1924af3bc0273c48386c96b466f2d3697823a0966ccb75712cd2fb-4ae46e0984ba5f14903914d5a25d1f403550735cbe46654dc97b79f27d088867 (173 of 173 patchlist). cut: /dev/stdin: No such file or directory Fetching 173 new ports or files... done. Removing old files and directories... done. Extracting new files: /usr/ports/MOVED /usr/ports/Mk/Scripts/ /usr/ports/Mk/Uses/ /usr/ports/UPDATING /usr/ports/archivers/pecl-lzf/ /usr/ports/astro/Makefile /usr/ports/astro/py-astropy-helpers/ files/7f8bf65c9fc6fac66ac8e2bbfbe5633ac1dfde7f9df090eb4dff5d646ec2d9ae.gz not found -- snapshot corrupt. There's definitely something going on here beyond just portsnap issues. Are you running inside a chroot or jail? Can you check that /dev/ is mounted? Which version of FreeBSD (especially the kernel) are you running? Although I tinkered around with jails a bit and may have some cbsd stuff in /etc/rc.conf.local, I'm not running any of these commands inside a chroot ot jail. I'm just running this as root on the system. This is the output of "mount" which shows that /dev is mounted: zroot/ROOT/default on / (zfs, local, noatime, nfsv4acls) devfs on /dev (devfs, local, multilabel) linprocfs on /compat/linux/proc (linprocfs, local) linsysfs on /compat/linux/sys (linsysfs, local) tmpfs on /compat/linux/dev/shm (tmpfs, local) fdescfs on /dev/fd (fdescfs) procfs on /proc (procfs, local) zroot/GoG on /GoG (zfs, local, noatime, nfsv4acls) This system was originally installed from a 12.1-RELEASE USB image, and I eventually migrated to 12-STABLE which I update from source roughly once I month, so I keep it pretty current. It's currently running 12.1-STABLE r363327 (kernel and world). But these issues with Portsnap were occurring even while I was running 12.1-RELEASE. I make sure to run "mergemaster -Ui", "make delete-old", and "make delete-old-libs" when I upgrade, so there shouldn't be any stale files causing issues. Hi Patrick, Can you check whether this issues goes away if you unmount /dev/fd? Most people don't have that, and devfs provides stdin/stdout and /dev/fd/[012] already; I'm wondering if fdescfs is breaking somehow. Colin Percival It doesn't appear to have made a difference. I ran "portsnap auto" for the first time today. It downloaded the latest patch, emitted errors about /dev/stdout, and failed with a "snapshot corrupt" message. I then unmounted /dev/fd and reran "portsnap auto" to see if anything changed. It didn't emit any errors about /dev/stdout on the subsequent attempt, but it still failed with "snapshot corrupt". I don't think the lack of messages about /dev/stdout on the subsequent attempt has anything with /dev/fd; I think they normally only appear the first time a patch is downloaded. As an aside, I have /dev/fd in my fstab because the installation message for OpenJDK says to put it there. Are the ports which direct users to add this to fstab incorrect and need be updated? Right, once portsnap gets into the state where it doesn't have the files it thinks it has, you'll get errors -- at that point the only thing you can do is get rid of /var/db/portsnap and start over. What I was wondering was whether unmounting fdescfs prevented the errors when you were first downloading the bits. The openjdk pkg-message is probably correct, so I'm not suggesting that you run without fdescfs in the long term -- I'm just wondering if this can be reproduced without fdescfs since knowing that will help us to pin down what's going wrong. Well, after last night, I deleted my Portsnap database and my ports tree, and I downloaded a fresh copy. Today, after seeing your reply, I removed fdescfs from fstab and rebooted so I know I'm working with a clean slate. I ran "portsnap auto" and updated the tree with no errors. Of course this error is intermittent (I went over two and a half months before it cropped up again), so I guess only time can tell for sure. as we' ve been added, how can port-secteam help here? The last weeks I notice the same issue. I have some jails and also I have /dev mounted as I need /dev/random /dev/mirror/gm0p2 on / (ufs, local, noatime) devfs on /dev (devfs, local, multilabel) /dev/mirror/gm0p7 on /home (ufs, local, noatime, nosuid, with quotas, soft-updates, acls) /dev/mirror/gm0p8 on /home2 (ufs, local, noatime, noexec, nosuid, soft-updates) /dev/mirror/gm0p4 on /tmp (ufs, local, noatime, nosuid, soft-updates) /dev/mirror/gm0p6 on /usr (ufs, local, noatime, soft-updates) /dev/mirror/gm0p5 on /var (ufs, local, noatime, soft-updates) tmpfs on /tmpfs (tmpfs, local, nosuid) /home/www on /home/jail/php72/home/www (nullfs, local, noatime, nosuid) /tmp on /home/jail/php72/tmp (nullfs, local, noatime, nosuid) /tmpfs on /home/jail/php72/tmpfs (nullfs, local, noatime, nosuid) devfs on /home/jail/php72/dev (devfs, local, multilabel) /home/www on /home/jail/php71/home/www (nullfs, local, noatime, nosuid) /tmp on /home/jail/php71/tmp (nullfs, local, noatime, nosuid) /tmpfs on /home/jail/php71/tmpfs (nullfs, local, noatime, nosuid) devfs on /home/jail/php71/dev (devfs, local, multilabel) /home/www on /home/jail/php56/home/www (nullfs, local, noatime, nosuid) /tmp on /home/jail/php56/tmp (nullfs, local, noatime, nosuid) /tmpfs on /home/jail/php56/tmpfs (nullfs, local, noatime, nosuid) devfs on /home/jail/php56/dev (devfs, local, multilabel) /tmp on /home/jail/redis/tmp (nullfs, local, noatime, nosuid) devfs on /home/jail/redis/dev (devfs, local, multilabel) I just wanted to follow up. I didn't realize it has been so long since I last posted, but it has been almost 5 months since I followed Colin's advice in comment 9 by unmounting /dev/fd, and the problem has not cropped up again. No activity for 3 years. I close here. If there are any questions or problems, feel free to reopen it. |