Bug 249166

Summary: ports-mgmt/pkg audit -F Segmentation fault - 1.15, regression
Product: Ports & Packages Reporter: Alexander Kuznetsov <alex>
Component: Individual Port(s)Assignee: freebsd-pkg (Nobody) <pkg>
Status: Closed FIXED    
Severity: Affects Many People CC: alex, asomers, bapt, davian818, mandree, portmgr, ports-secteam
Priority: --- Keywords: crash, regression
Version: LatestFlags: bugzilla: maintainer-feedback? (pkg)
Hardware: amd64   
OS: Any   
See Also: https://github.com/freebsd/pkg/issues/1878

Description Alexander Kuznetsov 2020-09-07 13:02:21 UTC
The latest version of pkg crashes on fetching vulnerabilities database.

# pkg -v
1.15.0
# pkg audit -F
Child process pid=55560 terminated abnormally: Segmentation fault
# pkg audit
0 problem(s) in 0 installed package(s) found.
# uname -r
12.1-RELEASE-p9

Previous build worked fine on the same system.

pkg was built from ports with poudriere.
Comment 1 Matthias Andree freebsd_committer freebsd_triage 2020-09-07 15:07:45 UTC
...its final words, with truss -f:

38964: openat(4,"local.conf",O_RDONLY,00)	 = 5 (0x5)
38964: fstat(5,{ mode=-rw-r--r-- ,inode=1163531,size=109,blksize=4096 }) = 0 (0x0)
38964: mmap(0x0,109,PROT_READ,MAP_SHARED,5,0x0)	 = 34376097792 (0x800f9a000)
38964: munmap(0x800f9a000,109)			 = 0 (0x0)
38964: close(5)					 = 0 (0x0)
38964: close(4)					 = 0 (0x0)
38964: openat(AT_FDCWD,"/var/db/pkg",O_RDONLY|O_DIRECTORY|O_CLOEXEC,00) = 4 (0x4)
38964: fstatat(4,"vuln.xml",{ mode=-r--r--r-- ,inode=11,size=6339069,blksize=131072 },0x0) = 0 (0x0)
38964: getrandom("(garbled stuff here)"...,40,0) = 40 (0x28)
38964: mmap(0x0,1104,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_ANON,-1,0x0) = 34376097792 (0x800f9a000)
38964: minherit(0x800f9a000,1104,INHERIT_ZERO)	 = 0 (0x0)
38964: fstatat(AT_FDCWD,"/tmp",{ mode=drwxrwxrwt ,inode=4,size=18,blksize=16384 },0x0) = 0 (0x0)
38964: open("/tmp/vuln.xml.bz2.FlL4hBhW",O_RDWR|O_CREAT|O_EXCL,0600) = 5 (0x5)
38964: SIGNAL 11 (SIGSEGV) code=SEGV_MAPERR trapno=12 addr=0x20
38964: process killed, signal = 11 (core dumped)

(gdb) bt
#0  0x000000000049775a in pkg_fetch_file_to_fd ()
#1  0x000000000049765d in pkg_fetch_file_tmp ()
#2  0x000000000044cb1b in pkg_audit_fetch ()
#3  0x000000000029faaa in exec_audit ()
#4  0x00000000002a8bbc in main ()
Comment 2 Matthias Andree freebsd_committer freebsd_triage 2020-09-07 15:10:41 UTC
Thread 1 (LWP 100699 of process 42411):
#0  0x000000000050413f in pkg_fetch_file_to_fd (repo=0x0, url=0x800fe7090 "http://vuxml.freebsd.org/freebsd/vuln.xml.bz2", dest=5, t=0x7fffffffcc38, offset=0, size=-1, silent=false) at fetch.c:226
        u = 0x0
        kv = 0x0
        kvtmp = 0x0
        envtorestore = 0x0
        envtounset = 0x0
        tmp = 0x0
        done = 0
        r = 0
        buf = '\000' <repeats 2024 times>...
        retcode = 0
        sz = 0
        buflen = 0
        left = 0
        fetcher = 0x0
        remote = 0x0
#1  0x0000000000503f35 in pkg_fetch_file_tmp (repo=0x0, url=0x800fe7090 "http://vuxml.freebsd.org/freebsd/vuln.xml.bz2", dest=0x7fffffffcde0 "/tmp/vuln.xml.bz2.I1CiSa8L", t=1599442709) at fetch.c:112
        fd = 5
        retcode = 3
#2  0x000000000047f2b9 in pkg_audit_fetch (src=0x800fe7090 "http://vuxml.freebsd.org/freebsd/vuln.xml.bz2", dest=0x0) at pkg_audit.c:276
        fd = -1
        outfd = -1
        tmp = "/tmp/vuln.xml.bz2.I1CiSa8L\000\000\000\000\000\000q\000\000\000q\203&5\000\206V\000\b\000\000\000\000\206V\000\b\000\000\000\a\000\000\000\000\000\000\000\203H#\000\000\000\000\000\200\200V\000\b\000\000\000\000\206V\000\b\000\000\000e[S\000\b\000\000\000\203H#\000\000\000\000\000\200\317\377\377\377\177\000\000࠶\000\b\000\000\000\a\000\000\000\b\000\000\000G\t\000\000\000\000\000\000TU\267\000\b\000\000\000\330\316\377\377\377\177\000\000\326\f\a\003^\254\203\245\300\316\377\377\377\177\000\000\344\066@\000\000\000\000\000\300\021\376\000\b\000\000\000\000\000\000\000\001\000\000\000\300\021\376\000\b\000\000\000\203H"...
        tmpdir = 0x240280 "/tmp"
        retcode = 3
        t = 1599442709
        st = {st_dev = 13574555021139550870, st_ino = 11, st_nlink = 1, st_mode = 33060, st_padding0 = 0, st_uid = 0, st_gid = 0, st_padding1 = 0, st_rdev = 18446744073709551615, st_atim = {tv_sec = 1599442709, tv_nsec = 659811000}, st_mtim = {tv_sec = 1599442709, tv_nsec = 306768000}, st_ctim = {tv_sec = 1599442709, tv_nsec = 306768000}, st_birthtim = {tv_sec = 1516670331, tv_nsec = 88524000}, st_size = 6339069, st_blocks = 7144, st_blksize = 131072, st_flags = 2048, st_gen = 8013493, st_spare = {0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}
        cbdata = {out = 1599491381, fname = 0x14 <error: Cannot access memory at address 0x14>, dest = 0x225a58 "CASE_SENSITIVE_MATCH"}
        dfd = 4
#3  0x00000000002ada78 in exec_audit (argc=0, argv=0x7fffffffdac8) at audit.c:164
        audit = 0x800fe72a0
        db = 0x0
        it = 0x0
        pkg = 0x0
        name = 0x7fffffffd310 "\346\333#"
        version = 0x800fe7030 "`p\376"
        audit_file = 0x0
        affected = 0
        vuln = 0
        fetch = true
        recursive = false
        ch = -1
        i = -11648
        ret = 0
        sb = 0x225a58
        check = 0x0
        longopts = {{name = 0x22e1b6 "fetch", has_arg = 0, flag = 0x0, val = 70}, {name = 0x247986 "file", has_arg = 1, flag = 0x0, val = 102}, {name = 0x23dbe6 "recursive", has_arg = 0, flag = 0x0, val = 114}, {name = 0x23521a "quiet", has_arg = 0, flag = 0x0, val = 113}, {name = 0x0, has_arg = 0, flag = 0x0, val = 0}}
#4  0x00000000002bc1ed in main (argc=2, argv=0x7fffffffdab8) at main.c:886
        i = 3
        command = 0x5220c0 <cmd+96>
        ambiguous = 0
        chroot_path = 0x0
        rootdir = 0x0
        jid = 0
        jail_str = 0x0
        len = 5
        ch = -1 '\377'
        debug = 0
        version = 0
        ret = 0
        plugins_enabled = true
        plugin_found = false
        show_commands = false
        activation_test = false
        init_flags = (unknown: 0)
        c = 0x0
        conffile = 0x0
        reposdir = 0x0
        save_argv = 0x7fffffffdab8
        realrootdir = "\000\000\000\000\000\000\000\000p\330\377\377\377\177", '\000' <repeats 11 times>, "\232T\000\b\000\000\000\000\327\377\377\377\177\000\000\067SS\000\b", '\000' <repeats 19 times>, "\271+!\000\000\000\000\000\004ϊ\006\000\000\000\000\364\362\217\362\000\000\000\000\060\002U\000\b\000\000\000\001\000\000\000\000\000\000\000\000HU\000\b\000\000\000\300{\266\000\b\000\000\000 \331\377\377\377\177\000\000\271+!\000\000\000\000\000\004ϊ\006\000\000\000\000\060\002U\000\b\000\000\000 \331\377\377\377\177\000\000\364\362\217\362\001\000\000\000\020\327\377\377\377\177\000\000\002\000\000\000\000\000\000\000\000\260T\000\b\000\000\000\000\260T\000\b\000\000\000p\330"...
        j = 0
        longopts = {{name = 0x22e2ac "debug", has_arg = 0, flag = 0x0, val = 100}, {name = 0x2202b5 "jail", has_arg = 1, flag = 0x0, val = 106}, {name = 0x2325e4 "chroot", has_arg = 1, flag = 0x0, val = 99}, {name = 0x221e5f "config", has_arg = 1, flag = 0x0, val = 67}, {name = 0x228873 "repo-conf-dir", has_arg = 1, flag = 0x0, val = 82}, {name = 0x24437d "rootdir", has_arg = 1, flag = 0x0, val = 114}, {name = 0x2201b7 "list", has_arg = 0, flag = 0x0, val = 108}, {name = 0x22cd3d "version", has_arg = 0, flag = 0x0, val = 118}, {name = 0x2460b5 "option", has_arg = 1, flag = 0x0, val = 111}, {name = 0x240375 "only-ipv4", has_arg = 0, flag = 0x0, val = 52}, {name = 0x221e66 "only-ipv6", has_arg = 0, flag = 0x0, val = 54}, {name = 0x0, has_arg = 0, flag = 0x0, val = 0}}
Comment 3 Matthias Andree freebsd_committer freebsd_triage 2020-09-07 15:11:10 UTC
221			}
222	
223			url += strlen(URL_SCHEME_PREFIX);
224			u = fetchParseURL(url);
225		} else {
226			if (repo->mirror_type == SRV && (strncmp(u->scheme, "http", 4) == 0 ||
227			    strncmp(u->scheme, "ftp", 3) == 0)) {
228				pkg_emit_notice(
229	     "Warning: use of %s:// URL scheme with SRV records is deprecated: "
230	     "switch to pkg+%s://", u->scheme, u->scheme);
Comment 4 Matthias Andree freebsd_committer freebsd_triage 2020-09-07 15:30:17 UTC
Adding Github issue (-> "see also" field) -- 
git bisect ends up here:

21a67b1f5e051de331f276310dab4976814abc79 is the first bad commit
commit 21a67b1f5e051de331f276310dab4976814abc79
Author: Baptiste Daroussin <bapt@FreeBSD.org>
Date:   Thu Apr 30 09:31:36 2020 +0200

    In case we do find the http mirror at full doc url path,
    Consider the file to fetch relatively to it

 libpkg/fetch.c       | 27 ++++++++++++++++++++-------
 libpkg/private/pkg.h |  1 +
 2 files changed, 21 insertions(+), 7 deletions(-)
Comment 5 Baptiste Daroussin freebsd_committer freebsd_triage 2020-09-07 16:01:34 UTC
fixed in 1.15.1
Comment 6 Urmas 2020-09-08 03:02:34 UTC
Still crashes, just a little later:

write(1,"Fetching vuln.xml.bz2:   0%",27)	 = 27 (0x1b)
SIGNAL 11 (SIGSEGV) code=SEGV_MAPERR trapno=12 addr=0xac
Comment 7 Matthias Andree freebsd_committer freebsd_triage 2020-09-08 13:53:45 UTC
Urmas, 

the report is useless and you are reporting a different crash than Alexander.

Please rebuild the new pkg 1.15.1 with WITH_DEBUG=yes set, then make it crash under gdb, and provide a backtrace in a *new* bug report.

Something along the lines of:

portsnap fetch update
cd /usr/ports/ports-mgmt/pkg
env WITH_DEBUG=yes make clean reinstall
gdb --args pkg-static (whatever other options you need)
run
<wait for crash>
<possibly> set pagination off                    
bt full
<post result to a new bug>

be sure to start the Summary line with    ports-mgmt/pkg:
so it gets auto-assigned.
Comment 8 Alan Somers freebsd_committer freebsd_triage 2020-09-14 17:50:27 UTC
I don't know if it's the same as what Urmass saw, but I get the following under pkg 1.15.1.
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=249323
Comment 9 Steve Wills freebsd_committer freebsd_triage 2020-09-14 17:53:09 UTC
*** Bug 249323 has been marked as a duplicate of this bug. ***
Comment 10 Alan Somers freebsd_committer freebsd_triage 2020-09-14 18:11:49 UTC
Steve, I opened a separate bug because as Matthias said, the new crash is not exactly the same as the old one.  And note that the new crash is on 1.15.1, which supposedly already contains the fix for this issue.
Comment 11 Matthias Andree freebsd_committer freebsd_triage 2020-09-14 18:59:23 UTC
Alan, I haven't been paying attention for a few days, and zap, I see pkg 1.15.4 is out... I wonder if pkg 1.15 is jinxed somehow though.

I'll restrain myself to just posting a link to the NEWS file
https://github.com/freebsd/pkg/blob/release-1.15/NEWS