Bug 208408

Summary: ports-mgmt/poudriere-devel : Unable to extract pkg
Product: Ports & Packages Reporter: korsani
Component: Individual Port(s)Assignee: Bryan Drewery <bdrewery>
Status: Open ---    
Severity: Affects Some People CC: dch, dvl, freebsd, grahamperrin, thierry, vsasjason, w.schwarzenfeld
Priority: --- Flags: bugzilla: maintainer-feedback? (bdrewery)
Version: Latest   
Hardware: Any   
OS: Any   
Attachments:
Description Flags
poudriere testport -o ports-mgmt/pkg -j 10-3 none

Description korsani 2016-03-30 20:45:18 UTC 
Created attachment 168801 [details]
poudriere testport -o ports-mgmt/pkg -j 10-3

Hi,
I've updated my poudriere's jail from 10.3-RC3 to 10.3-RELEASE and I got "Unable to extract pkg" when building ports-mgmt/pkg.
I destroyed ports and jail to be safe: same message
I join the log made with poudriere testport (in which this message is absent)

Here is my poudriere.conf;

NO_ZFS=yes
FREEBSD_HOST=https://download.freebsd.org
FREEBSD_HOST=ftp://ftp5.fr.freebsd.org
RESOLV_CONF=/etc/resolv.conf
BASEFS=/mnt/space/1/poudriere
USE_PORTLINT=no
USE_TMPFS=yes
DISTFILES_CACHE=/mnt/space/1/distfiles
SVN_HOST=svn.FreeBSD.org
CHECK_CHANGED_OPTIONS=verbose
CHECK_CHANGED_DEPS=yes
NOLINUX=yes
NO_LINUX=yes
ALLOW_MAKE_JOBS=yes
ALLOW_MAKE_JOBS_PACKAGES="pkg ccache py*"

And the log when bulking package :
# poudriere bulk -vv -j 10-3 -p default ports-mgmt/pkg
[00:00:00] ====>> Creating the reference jail... done
[00:00:27] ====>> Mounting system devices for 10-3-default
[00:00:27] ====>> Mounting ports/packages/distfiles
[00:00:27] ====>> Using packages from previously failed build
[00:00:27] ====>> Mounting packages from: /mnt/space/1/poudriere/data/packages/10-3-default
/etc/resolv.conf -> /mnt/space/1/poudriere/data/.m/10-3-default/ref/etc/resolv.conf
[00:00:28] ====>> Starting jail 10-3-default
[00:00:28] ====>> Logs: /mnt/space/1/poudriere/data/logs/bulk/10-3-default/2016-03-30_22h41m34s
[00:00:28] ====>> Loading MOVED
[00:00:29] ====>> Calculating ports order and dependencies
[00:00:29] ====>> Computing deps for ports-mgmt/pkg
[00:00:29] ====>> pkg package missing, skipping sanity
[00:00:30] ====>> Skipping incremental rebuild and repository sanity checks
[00:00:30] ====>> Cleaning the build queue
[00:00:30] ====>> Recording filesystem state for prepkg... done
[00:00:32] ====>> Building 1 packages using 1 builders
[00:00:32] ====>> Starting/Cloning builders
[00:00:52] ====>> Hit CTRL+t at any time to see build progress and stats
[00:00:52] ====>> Stopping 1 builders
[00:00:55] ====>> No package built, but repository needs to be created
[00:00:55] ====>> Creating pkgng repository
[00:00:55] ====>> Error: Unable to extract pkg.
[00:00:55] ====>> Cleaning up
[00:00:55] ====>> Umounting file systems


I didn't set any "poudriere option" for ports-mgmt/pkg

Any clue ?
Comment 1 korsani 2016-04-06 00:27:11 UTC 
Update: 
After setting some set -x to /usr/local/share/poudriere/common.sh, I found that this line fails :
+ [ -e /mnt/space/3/poudriere/data/.m/zon-default/ref/packages/Latest/pkg.txz ]
Which is in function :

ensure_pkg_installed() {
    local force="$1"
    local mnt

    _my_path mnt
    [ ${PKGNG} -eq 1 ] || return 0
    [ -z "${force}" ] && [ -x "${mnt}${PKG_BIN}" ] && return 0
    # Hack, speed up QEMU usage on pkg-repo.
    if [ ${QEMU_EMULATING:-0} -eq 1 ] && \
        [ -f /usr/local/sbin/pkg-static ]; then
        cp -f /usr/local/sbin/pkg-static "${mnt}/.p/pkg-static"
        return 0
    fi
    [ -e ${MASTERMNT}/packages/Latest/pkg.txz ] || return 1 #pkg missing
    injail tar xf /packages/Latest/pkg.txz -C / \
        -s ",/.*/,.p/,g" "*/pkg-static"
    return 0
}

At line marked "pkg missing"

In my case, ${MASTERMNT}/packages/Latest/pkg.txz is resolved in:
/mnt/space/3/poudriere/data/packages/zon-default/.building  641397412    8140044 581945576     1%    /mnt/space/3/poudriere/data/.m/zon-default/ref/packages

and former directory is always empty

Did I made something wrong for that directory beeing empty ? and/or pkg not beeing built ?
Comment 2 Walter Schwarzenfeld freebsd_triage 2018-01-13 03:59:07 UTC 
Maintainer feedback?
Comment 3 Bryan Drewery freebsd_committer freebsd_triage 2018-01-24 19:45:31 UTC 
This was fixed quite a while ago.
Comment 4 Dan Langille freebsd_committer freebsd_triage 2020-02-04 02:05:23 UTC 
No, it was not fixed. Stil there today on FreeBSD 12.1 with poudriere-3.3.3
Comment 5 Thierry Thomas freebsd_committer freebsd_triage 2020-06-19 18:27:22 UTC 
(In reply to Dan Langille from comment #4)

Same here.
Comment 6 Peter 2020-08-26 15:19:12 UTC 
Same here.
.latest/Latest/ directory was empty, so: pkg package missing, skipping sanity

manual work around was:
:# cp All/pkg-1.14.7.txz .latest/Latest/pkg.txz

after I did that poudriere is functional again.
Comment 7 Dan Langille freebsd_committer freebsd_triage 2020-10-16 19:16:17 UTC 
Just hit it with 2020Q4

I suspect it is related to a failed build.
Comment 8 Dan Langille freebsd_committer freebsd_triage 2020-10-16 19:18:43 UTC 
(In reply to Dan Langille from comment #7)
This fixed it for me:

[dvl@ava-pkg-01prd:/usr/local/poudriere/data/packages/FreeBSD:12:amd64-2020Q4] $ sudo rm -rf .building
Comment 9 Dan Langille freebsd_committer freebsd_triage 2020-12-16 22:49:38 UTC 
I'm hitting this with poudriere-3.3.6 and it's blocking a production fix.
Comment 10 Dan Langille freebsd_committer freebsd_triage 2020-12-16 23:01:06 UTC 
This fixed it this time, run poudriere with:

*  -C ports-mgmt/pkg
* security/clamav (what I needed to build, but it built only ports-mgmt/pkg)
* -C security/clamav

Success!
Comment 11 Dave Cottlehuber freebsd_committer freebsd_triage 2022-11-29 22:46:01 UTC 
seeing this on fresh VM, 13.1-RELEASE amd64, with a custom 13.1-RELEASE+patches jail, and a default ports tree, using poudriere-devel-3.3.99.20220831

It appears (as reported by dvl), that running a `poudriere bulk -C ...` prior
is sufficient to fix this.