Bug 253292 - regression in r550860 (@sample conversion to lua) semantic change causes leftovers in poudriere, ex: security/ca_root_nss
Summary: regression in r550860 (@sample conversion to lua) semantic change causes left...
Status: Closed FIXED
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Ports Framework (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Only Me
Assignee: Port Management Team
URL:
Keywords: regression
Depends on:
Blocks: 250439
  Show dependency treegraph
 
Reported: 2021-02-06 11:41 UTC by Matthias Andree
Modified: 2021-02-26 15:43 UTC (History)
5 users (show)

See Also:
mandree: merge-quarterly+


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Matthias Andree freebsd_committer freebsd_triage 2021-02-06 11:41:41 UTC
Greetings,

I am debugging a leftovers situation in security/ca_root_nss. Turns out that since the Lua conversion, @sample (Keywords/sample.ucl) has different semantic than the Shell version used to have.

I am looking at pkg 1.6.2, ca_root_nss 3.5.8, and ports from SVN at r564205 (that's the ^/head ummmm... trunk I'd say).

Here's how, when ca_root_nss's pre-deinstall script runs:

Situation:
1. pkg-plist contains:
@sample etc/ssl/cert.pem.sample

2. symlinks at deinstall time, after fresh installation of ca_root_nss package

$ ls -l /usr/local/etc/ssl/cert.pem.sample /usr/local/etc/ssl/cert.pem
lrwxr-xr-x  1 root  wheel  33 30 Jan. 02:23 /usr/local/etc/ssl/cert.pem.sample -> ../../share/certs/ca-root-nss.crt
-rw-r--r--  1 root  wheel  786736 17 Okt. 18:23 /usr/local/etc/ssl/cert.pem

3. tracing with gdb into lua_pkg_filecmp() (you need to "set follow-fork-mode child"):
(gdb) 
198		const char* file1 = luaL_checkstring(L, 1);
(gdb) 
199		const char* file2 = luaL_checkstring(L, 2);
(gdb) print file1
$1 = 0x80109b618 "/usr/local/etc/ssl/cert.pem.sample"
(gdb) print file2
$2 = 0x80109b758 "/usr/local/etc/ssl/cert.pem"
...
208		if (fstatat(pkg->rootfd, RELATIVE_PATH(file1), &s1, AT_SYMLINK_NOFOLLOW) == -1) {
(gdb) 
212		if (fstatat(pkg->rootfd, RELATIVE_PATH(file2), &s2, AT_SYMLINK_NOFOLLOW) == -1) {
(gdb) 
216		if (!S_ISREG(s1.st_mode) || !S_ISREG(s2.st_mode)) {
(gdb) 
217			lua_pushinteger(L, -1);

so it errors out here because file1 isn't regular.

HOW IS THIS DIFFERENT?
1) cmp -s in the earlier shell version of the script didn't care if it was looking at regular files or symlinks or whatnot but would just open and compare contents and exit 0 (same content)
2) the lua version now ERRORS out (-1) (and @sample ignores that and just leaves the file) because it isn't looking at two regular files.

To me, it is not clear why pkg's lua_pkg_filecmp() cares so much about file type WITHOUT following symlink. If it were, as a fallback, comparing symlinks, that might have a selling point, but the way things are in pkg 1.6.2, it's not clear to me.  Might rather be a quick sanity check (is the output something we can mmap()) that misfires in corner cases. such as this.


OPTIONS:
1. change pkg's lua_pkg_filecmp() to follow symlinks or disregard unimportant file type differences. Plus: keeps capsicum, fewer external commands.
2. revert the switch from shell to lua script made in ports r550860. Plus: reinstates former behavior. Minus: loses capsicum isolation.
3. patch ALL ports that mix symlinks with @sample. Minus: doesn't scale.

I am proposing (1), i. e. bringing pkg closer to former cmp behavior.
Comment 1 Matthias Andree freebsd_committer freebsd_triage 2021-02-06 11:42:30 UTC
(the leftovers appear in poudriere bulk -t or poudriere testport, but will appear in practice, too)
Comment 2 Vladimir Druzenko freebsd_committer freebsd_triage 2021-02-07 08:29:35 UTC
(In reply to Matthias Andree from comment #0)
> 3. patch ALL ports that mix symlinks with @sample. Minus: doesn't scale.
Impossible. This error manifests itself if directory in path to install plalce is symlink too, not just sample files.
Comment 3 commit-hook freebsd_committer freebsd_triage 2021-02-18 18:05:43 UTC
A commit references this bug:

Author: manu
Date: Thu Feb 18 18:04:46 UTC 2021
New revision: 565957
URL: https://svnweb.freebsd.org/changeset/ports/565957

Log:
  ports-mgmt/pkg-devel: Update to 1.16.99.2

  - libpkg: rsa: stop leaking an RSA object
  - libpkg: rsa: start abstracting away rsa bits
  - lua: filecmp: Fix for symlink

  PR:		253292, 250439
  Approved by:	bapt (implicit)

Changes:
  head/ports-mgmt/pkg-devel/Makefile
  head/ports-mgmt/pkg-devel/distinfo
Comment 4 commit-hook freebsd_committer freebsd_triage 2021-02-18 18:11:48 UTC
A commit references this bug:

Author: manu
Date: Thu Feb 18 18:11:07 UTC 2021
New revision: 565958
URL: https://svnweb.freebsd.org/changeset/ports/565958

Log:
  ports-mgmt/pkg: Update to 1.16.3

  - lua: filecmp: Fix for symlink

  PR:	253292, 250439
  Approved by:	bapt (implicit)
  MFH:		2021Q1

Changes:
  head/ports-mgmt/pkg/Makefile
  head/ports-mgmt/pkg/distinfo
Comment 5 commit-hook freebsd_committer freebsd_triage 2021-02-18 18:13:54 UTC
A commit references this bug:

Author: manu
Date: Thu Feb 18 18:13:34 UTC 2021
New revision: 565960
URL: https://svnweb.freebsd.org/changeset/ports/565960

Log:
  MFH: r562140 r565958

  ports-mgmt/pkg: Update to 1.16.2

  Changes from 1.16.1 to 1.16.2
  - libpkg: add a snap(shot) version prefix
  - libpkg: only upgrade installed packages with pattern matches
  - Document pkg-lock(8) accepts a list of packages

  Approved by:	bapt (implicit)

  ports-mgmt/pkg: Update to 1.16.3

  - lua: filecmp: Fix for symlink

  PR:	253292, 250439
  Approved by:	bapt (implicit)

Changes:
_U  branches/2021Q1/
  branches/2021Q1/ports-mgmt/pkg/Makefile
  branches/2021Q1/ports-mgmt/pkg/distinfo
Comment 6 Vladimir Druzenko freebsd_committer freebsd_triage 2021-02-21 07:34:50 UTC
It's fixed now for my ports.
Comment 7 Matthias Andree freebsd_committer freebsd_triage 2021-02-21 17:31:49 UTC
pkg 1.16.3 appears to have fixed this.