It occurred to me that I could save oodles of disk space (and, for certain operations, speed things up) by using archivemount to mount compressed archives of /usr/ports and /usr/src. This is on FreeBSD 13.0 with the archivemount 0.8.12 package. I issued the command: archivemount -o readonly ports.txz /mnt Ports.txz being the release tarball for 13.0. An ls of /mnt/usr/ports gave the expected results. However, running du -s /mnt gave a long stream of "No such file or directory" error messages for items that actually existed if I cd'd and looked. I tried a gzipped version of the tarball. Same thing. So I tried an uncompressed tar file. Ditto. Then I zipped up /usr/ports and tried the zip file. Ditto. I tried the src tarball. Ditto. Then I tried the base tarball. A du -s /mnt gave a plausible result; I didn't check it. Because after I ran that command, all further commands returned "Input/output error". Ditto when I tried the uncompressed tarball. However, it did seem to behave with the kernel tarball. Next, I tarred the base, kernel, src, and ports tarball and tried that tar file. All good, so it isn't just a matter of the archive size. I tarred up /usr/ports/devel (the directory with the largest number of entries) and tried that. No problem, so it isn't just a matter of a large number of directory entries in a directory. I left it at that and prepared this PR.
In an attempt to further isolate the bug, I tried using a different package, fuse-zip, with a zipped up /usr/ports. And--got the same bug. No, the problem isn't in fuse itself, it's two separate bugs, one in fuse-zip and one in archivemount. In both cases, the link count for directories (and for archivemount at least, files), is computed incorrectly. For archivemount the link count is always returned as zero. For fuse-zip, the "link count" is actually the number of entries in the directory. Either way, the incorrect link count confuses the directory traversal code of fts.c because it uses that count to optimize away unnecessary stat calls. And this ultimately led du to generate the error messages that I saw. It also breaks find but, apparently, not diff -r. Be that as it may, this is a serious bug and needs some sort of repair. (Ditto for the bug in fuse-zip, though that's for a different PR.) And here is an awful hack to fix the bug. THIS IS NOT A CORRECT FIX. First, it always sets the link count to 1 for non-directories. Second, it is probably unnecessarily inefficient. But it suffices to set the link count for directories correctly, which makes du and find work. This patches archivemount.c: @@ -1545,6 +1545,20 @@ stbuf->st_blocks = (stbuf->st_size + 511) / 512; stbuf->st_blksize = 4096; + if (S_ISDIR(stbuf->st_mode)) { + stbuf->st_nlink = 2; + NODE *child, *tmp; + HASH_ITER(hh, node->child, child, tmp) { + const struct stat *chbuf = archive_entry_stat(child->entry); + if (S_ISDIR(chbuf->st_mode)) { + ++stbuf->st_nlink; + } + } + } else { + stbuf->st_nlink = 1; + } +
Could you check this error with new port version of archivemount (0.9.1) ?