Bug 252184 - sysutils/e2fsprogs: checksum mismatch
Summary: sysutils/e2fsprogs: checksum mismatch
Status: Closed FIXED
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: amd64 Any
: --- Affects Only Me
Assignee: Matthias Andree
URL:
Keywords:
Depends on: 257816 257817
Blocks:
  Show dependency treegraph
 
Reported: 2020-12-27 04:14 UTC by corvid
Modified: 2021-08-13 17:20 UTC (History)
4 users (show)

See Also:
mandree: maintainer-feedback+


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description corvid 2020-12-27 04:14:10 UTC
I tried removing and reinstalling e2fsprogs, and it made no difference.

# pkg check -s e2fsprogs
Checking e2fsprogs:   0%
e2fsprogs-1.45.6_5: checksum mismatch for /usr/local/sbin/e2fsck
e2fsprogs-1.45.6_5: checksum mismatch for /usr/local/sbin/fsck.ext2
e2fsprogs-1.45.6_5: checksum mismatch for /usr/local/sbin/fsck.ext3
e2fsprogs-1.45.6_5: checksum mismatch for /usr/local/sbin/fsck.ext4
e2fsprogs-1.45.6_5: checksum mismatch for /usr/local/sbin/fsck_ext2fs

$  uname -mrs
FreeBSD 12.2-RELEASE-p1 amd64

$ cat /usr/local/etc/pkg/repos/FreeBSD.conf 
FreeBSD: {
  url: "pkg+http://pkg.FreeBSD.org/${ABI}/latest",
}
Comment 1 Matthias Andree freebsd_committer freebsd_triage 2020-12-27 16:14:58 UTC
This is a known issue due to a pkg limitation.  

We need to have the same executable accessible in $PREFIX/sbin where it belongs, and under /sbin.  In order to avoid packaging the same file twice, I could use symlinks or hardlinks. Hardlinks would break when pkg unpacks the package on installation in case $PREFIX/sbin were not on the same file system as /sbin.

The current solution is that the post-install script (pkg-install) tries to convert them to hard links on systems where this is possible. This happens without pkg's knowing and causes the checksum differences.

I'd previously tried to just linking or copyoing the files without packaging them, which caused PRs complaining that "pkg which" would't report which package the files originated from...

So some artifact will remain until pkg learns some "hardlink or copy" option.
Comment 2 commit-hook freebsd_committer freebsd_triage 2021-08-04 21:23:52 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/ports/commit/?id=54b26298c82275501e6dcc6c733d21c0a6bf0d9f

commit 54b26298c82275501e6dcc6c733d21c0a6bf0d9f
Author:     Matthias Andree <mandree@FreeBSD.org>
AuthorDate: 2021-08-04 20:09:19 +0000
Commit:     Matthias Andree <mandree@FreeBSD.org>
CommitDate: 2021-08-04 21:23:08 +0000

    sysutils/e2fsprogs: port rework

    1 - sysutils/e2fsprogs: fix checksum mismatches

        e2fsprogs has replaced symlinks by hardlinks in its post-install if
        /sbin and ${PREFIX}/sbin were on the same file system, and unless
        script processing was defeated on install or upgrade.

        Leave symlinks in place to avoid checksum mismatches.

    PR:             252184 [1]

    2 - add FLAVORS for /sbin handling

        - no flavor (default): set symlinks from $PREFIX/sbin to /sbin
        - roothardlinks: set hardlinks from $PREFIX/sbin to /sbin, when
          $PREFIX, its children, and /sbin reside on the same file system
        - nobootfsck: do not install into /sbin, but this makes fsck
          unavailable for ext2/ext3/ext4 file systems before $PREFIX/ is
          mounted

    Require tests on i386 and amd64 only for FreeBSD >= 14 (not >= 12).

    Do not print removal advice from post-uninstall on upgrades.

 sysutils/e2fsprogs/Makefile    | 40 +++++++++++++++++++++++++++++++++++++---
 sysutils/e2fsprogs/pkg-install | 21 ++-------------------
 sysutils/e2fsprogs/pkg-plist   |  4 ++--
 3 files changed, 41 insertions(+), 24 deletions(-)
Comment 3 Serge Volkov 2021-08-12 21:18:43 UTC
I have the latest version of the e2fsprogs package installed. I tried transferring the system to another hard drive using dump/restore. After restore I have:

pkg check -s e2fsprogs
Checking e2fsprogs:   0%
e2fsprogs-1.46.3: missing file /usr/local/sbin/fsck.ext2
e2fsprogs-1.46.3: missing file /usr/local/sbin/fsck.ext3
e2fsprogs-1.46.3: missing file /usr/local/sbin/fsck.ext4
Checking e2fsprogs: 100%

Why is this happening? And how can I avoid this without reinstalling e2fsprogs?

--
FreeBSD 12.2 RELEASE p9 amd64.
Comment 4 Matthias Andree freebsd_committer freebsd_triage 2021-08-12 23:07:51 UTC
(In reply to Serge Volkov from comment #3)
Serge, I don't know how or why this happened.  Were there any errors or warnings reported from dump or restore?  What options did you use?  Did you restore into the same directories, or move things around?  What file systems types did you use?

This is an extract from a fresh package install:

1446205 lrwxr-xr-x  4 root  wheel      12 Aug  5 03:12 /usr/local/sbin/e2fsck -> /sbin/e2fsck
1446206 -r-xr-xr-x  1 root  wheel   32800 Aug  5 03:12 /usr/local/sbin/e2image
1446244 -r-xr-xr-x  3 root  wheel  104912 Aug  5 03:12 /usr/local/sbin/e2label
1446149 -r-xr-xr-x  2 root  wheel   25472 Aug  5 03:12 /usr/local/sbin/e2mmpstatus
1446245 -r-xr-xr-x  1 root  wheel   15704 Aug  5 03:12 /usr/local/sbin/e2undo
1446244 -r-xr-xr-x  3 root  wheel  104912 Aug  5 03:12 /usr/local/sbin/findfs
1446205 lrwxr-xr-x  4 root  wheel      12 Aug  5 03:12 /usr/local/sbin/fsck.ext2 -> /sbin/e2fsck
1446205 lrwxr-xr-x  4 root  wheel      12 Aug  5 03:12 /usr/local/sbin/fsck.ext3 -> /sbin/e2fsck
1446205 lrwxr-xr-x  4 root  wheel      12 Aug  5 03:12 /usr/local/sbin/fsck.ext4 -> /sbin/e2fsck


You can see that the missing fsck.ext* files are hard links with e2fsck which in itself is a symlink, as of 1.46.3. 

You can recreate them easily (or else you can reinstall e2fsprogs).

cd /usr/local/sbin
ln e2fsck fsck.ext2
ln e2fsck fsck.ext3
ln e2fsck fsck.ext4
Comment 5 Serge Volkov 2021-08-13 11:16:09 UTC
(In reply to Matthias Andree from comment #4)

For cloning full system I use

/sbin/mount_msdosfs /dev/da0s1 /mnt
/sbin/dump -0Lf - / | /usr/local/bin/7z a -si -mx=9 /mnt/dump.img.7z >/dev/null
/sbin/umount /mnt

No errors occur during the dumping.

For restore on another hard drive I use

newfs -U /dev/ada1p3
tunefs -j disable /dev/ada1p3
mkdir -p /mnt
mount /dev/ada1p3 /mnt
cd /mnt && 7z x -so dump.img.7z | restore -vrf -

Here is part of the restore log:

...
extract file ./usr/local/include/tss/tss_structs.h
Create symbolic link ./usr/local/sbin/e2fsck->/sbin/e2fsck
extract file ./usr/local/sbin/e2image
...
Create hard link ./usr/local/share/poppler/cMap/Adobe-Japan1/UniJIS-UTF32-H->./usr/local/share/cmap/UniJIS-UTF32-H
Create hard link ./usr/local/sharewarning: cannot create hard link ./usr/local/sbin/fsck.ext4->./usr/local/sbin/e2fsck: No such file or directory
warning: cannot create hard link ./usr/local/sbin/fsck.ext3->./usr/local/sbin/e2fsck: No such file or directory
warning: cannot create hard link ./usr/local/sbin/fsck.ext2->./usr/local/sbin/e2fsck: No such file or directory
/poppler/cMap/Adobe-Japan1/UniJIS-UTF32-V->./usr/local/share/cmap/UniJIS-UTF32-V
Create hard link ./usr/local/share/poppler/cMap/Adobe-Japan1/UniJIS-UTF8-H->./usr/local/share/cmap/UniJIS-UTF8-H
...

I use only UFS file system. As you can see, the error occurs when trying to create a hard links. I often need to clone one system across multiple disks. Therefore, it is not convenient to reinstall the e2fsprogs package on each new disk.

If I use your advice and create links, I get the following:

pkg check -s e2fsprogs
Checking e2fsprogs:   0%
e2fsprogs-1.43.3: checksum mismatch for /usr/local/sbin/fsck.ext2
e2fsprogs-1.46.3: checksum mismatch for /usr/local/sbin/fsck.ext3
e2fsprogs-1.46.3: checksum mismatch for /usr/local/sbin/fsck.ext4

Is there any other way to solve this problem?
Comment 6 Matthias Andree freebsd_committer freebsd_triage 2021-08-13 17:08:27 UTC
Oh joy. After some experimentation, I conclude that restore(8) and tar(1) are broken on FreeBSD 13.0-RELEASE-p3. Neither can restore the hardlink-to-absolute-symlink.

Use gtar -H posix to create the "backup" (posix for nanosecond timestamps) and extract it with gtar. Not sure if that copies flags.

With respect to the bad checksums, my apologies for providing inaccurate information. Please do:

cd /usr/local/sbin
ln -fhP e2fsck fsck.ext2
ln -fhP e2fsck fsck.ext3
ln -fhP e2fsck fsck.ext4