Bug 210888

Summary: ports-mgmt/pkg: pkg 1.8.6 loops while upgrading itself from local repository
Product: Ports & Packages Reporter: Axel.Rau
Component: Individual Port(s)Assignee: freebsd-pkg (Nobody) <pkg>
Status: Closed Overcome By Events    
Severity: Affects Only Me CC: pkg, w.schwarzenfeld
Priority: --- Keywords: needs-qa, regression
Version: LatestFlags: koobs: maintainer-feedback? (pkg)
Hardware: Any   
OS: Any   

Description Axel.Rau 2016-07-07 11:47:42 UTC
Upgrading pkg from local repository loops, creating child processes until process limit reached:
- - -
root@hermes:~ #	pkg upgrade ports-mgmt/pkg
Updating lrau-repo-py3 repository catalogue...
lrau-repo-py3 repository is up-to-date.
Updating lrau-repo repository catalogue...
lrau-repo repository is up-to-date.
All repositories are up-to-date.
New version of pkg detected; it needs to be installed first.
Checking integrity... done (0 conflicting)
Your packages are up to date.
Updating lrau-repo-py3 repository catalogue...
lrau-repo-py3 repository is up-to-date.
Updating lrau-repo repository catalogue...
lrau-repo repository is up-to-date.
All repositories are up-to-date.
. . .

Upgrading from FreeBSD repository works as expected.

Config:

root@hermes:~ #	uname -a
FreeBSD hermes.in.chaos1.de 10.2-RELEASE-p18 FreeBSD 10.2-RELEASE-p18 #0: Sat May 28 08:53:43 UTC 2016     root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC  amd64
root@hermes:~ #	pkg -vv
Version                 : 1.8.6
PKG_DBDIR = "/var/db/pkg";
PKG_CACHEDIR = "/var/cache/pkg";
PORTSDIR = "/usr/ports";
INDEXDIR = "";
INDEXFILE = "INDEX-10";
HANDLE_RC_SCRIPTS = false;
DEFAULT_ALWAYS_YES = false;
ASSUME_ALWAYS_YES = false;
REPOS_DIR [
    "/etc/pkg/",
    "/usr/local/etc/pkg/repos/",
]
PLIST_KEYWORDS_DIR = "";
SYSLOG = true;
ABI = "FreeBSD:10:amd64";
ALTABI = "freebsd:10:x86:64";
DEVELOPER_MODE = false;
VULNXML_SITE = "http://vuxml.freebsd.org/freebsd/vuln.xml.bz2";
FETCH_RETRY = 3;
PKG_PLUGINS_DIR = "/usr/local/lib/pkg/";
PKG_ENABLE_PLUGINS = true;
PLUGINS [
]
DEBUG_SCRIPTS = false;
PLUGINS_CONF_DIR = "/usr/local/etc/pkg/";
PERMISSIVE = false;
REPO_AUTOUPDATE = true;
NAMESERVER = "";
HTTP_USER_AGENT = "pkg/1.8.6";
EVENT_PIPE = "";
FETCH_TIMEOUT = 30;
UNSET_TIMESTAMP = false;
SSH_RESTRICT_DIR = "";
PKG_ENV {
}
PKG_SSH_ARGS = "";
DEBUG_LEVEL = 0;
ALIAS {
    all-depends = "query %dn-%dv";
    annotations = "info -A";
    build-depends = "info -qd";
    cinfo = "info -Cx";
    comment = "query -i \"%c\"";
    csearch = "search -Cx";
    desc = "query -i \"%e\"";
    download = "fetch";
    iinfo = "info -ix";
    isearch = "search -ix";
    prime-list = "query -e '%a = 0' '%n'";
    leaf = "query -e '%#r == 0' '%n-%v'";
    list = "info -ql";
    noauto = "query -e '%a == 0' '%n-%v'";
    options = "query -i \"%n - %Ok: %Ov\"";
    origin = "info -qo";
    provided-depends = "info -qb";
    raw = "info -R";
    required-depends = "info -qr";
    roptions = "rquery -i \"%n - %Ok: %Ov\"";
    shared-depends = "info -qB";
    show = "info -f -k";
    size = "info -sq";
}
CUDF_SOLVER = "";
SAT_SOLVER = "";
RUN_SCRIPTS = true;
CASE_SENSITIVE_MATCH = false;
LOCK_WAIT = 1;
LOCK_RETRIES = 5;
SQLITE_PROFILE = false;
WORKERS_COUNT = 0;
READ_LOCK = false;
PLIST_ACCEPT_DIRECTORIES = false;
IP_VERSION = 0;
AUTOMERGE = true;
VERSION_SOURCE = "";
CONSERVATIVE_UPGRADE = true;
PKG_CREATE_VERBOSE = false;
AUTOCLEAN = false;
DOT_FILE = "";
REPOSITORIES {
}
VALID_URL_SCHEME [
    "pkg+http",
    "pkg+https",
    "https",
    "http",
    "file",
    "ssh",
    "ftp",
    "ftps",
    "pkg+ssh",
    "pkg+ftp",
    "pkg+ftps",
]
ALLOW_BASE_SHLIBS = false;
WARN_SIZE_LIMIT = 1048576;


Repositories:
  lrau-repo-py3: { 
    url             : "http://db1.in.chaos1.de/102amd64-default-py3",
    enabled         : yes,
    priority        : 0,
    mirror_type     : "HTTP",
    pubkey          : "/usr/local/etc/ssl/certs/pkg.cert"
  }
  lrau-repo: { 
    url             : "http://db1.in.chaos1.de/102amd64-default",
    enabled         : yes,
    priority        : 0,
    mirror_type     : "HTTP"
  }
root@hermes:~ #	

with debug turned on:

root@hermes:~ #	pkg -d -d -d  upgrade ports-mgmt/pkg
DBG(1)[26024]> pkg initialized
Updating lrau-repo-py3 repository catalogue...
DBG(1)[26024]> PkgRepo: verifying update for lrau-repo-py3
DBG(1)[26024]> Pkgrepo, begin update of '/var/db/pkg/repo-lrau-repo-py3.sqlite'
DBG(1)[26024]> Fetch: fetching from: http://db1.in.chaos1.de/102amd64-default-py3/meta.txz with opts "i"
DBG(1)[26024]> Fetch: fetching from: http://db1.in.chaos1.de/102amd64-default-py3/packagesite.txz with opts "i"
lrau-repo-py3 repository is up-to-date.
Updating lrau-repo repository catalogue...
DBG(1)[26024]> PkgRepo: verifying update for lrau-repo
DBG(1)[26024]> Pkgrepo, begin update of '/var/db/pkg/repo-lrau-repo.sqlite'
DBG(1)[26024]> Fetch: fetching from: http://db1.in.chaos1.de/102amd64-default/meta.txz with opts "i"
DBG(1)[26024]> Fetch: fetching from: http://db1.in.chaos1.de/102amd64-default/packagesite.txz with opts "i"
lrau-repo repository is up-to-date.
All repositories are up-to-date.
DBG(1)[26024]> want to get an advisory lock on a database
DBG(3)[26024]> added shlib provide libpkg.so.3 for pkg
DBG(3)[26024]> added shlib provide libpkg.so.3 for pkg
DBG(3)[26024]> added shlib provide libpkg.so.3 for pkg
DBG(3)[26024]> added shlib provide libpkg.so.3 for pkg
DBG(3)[26024]> added shlib provide libpkg.so.3 for pkg
DBG(2)[26024]> universe: add new local pkg: pkg, (pkg-1.8.6:2$0$y73z87qnfy46xnsn1nr9a5ioch916y5rxdkrkbcshj5p6ci5txqb)
DBG(2)[26024]> universe: add new remote pkg: pkg, (pkg-1.8.7:2$0$ctuxksb3do8t4x5jqomed5ertj46p338smopp8pe1uyumg3g4enb)
DBG(2)[26024]> universe: add new remote pkg: pkg, (pkg-1.8.7:2$0$ctuxksb3do8t4x5jqomed5ertj46p338smopp8pe1uyumg3g4enb)
DBG(3)[26024]> added shlib provide libpkg.so.3 for pkg
New version of pkg detected; it needs to be installed first.
DBG(1)[26024]> removing pkg from the request as it is the same as local
DBG(2)[26024]> upgrade rule: upgrade local pkg-1.8.7 to remote pkg-1.8.7
DBG(2)[26024]> upgrade rule: upgrade local pkg-1.8.6 to remote pkg-1.8.7
DBG(2)[26024]> upgrade rule: upgrade local pkg-1.8.6 to remote pkg-1.8.7
DBG(2)[26024]> decided local pkg-2$0$y73z87qnfy46xnsn1nr9a5ioch916y5rxdkrkbcshj5p6ci5txqb to install
DBG(2)[26024]> decided remote pkg-2$0$ctuxksb3do8t4x5jqomed5ertj46p338smopp8pe1uyumg3g4enb to delete
DBG(2)[26024]> decided remote pkg-2$0$ctuxksb3do8t4x5jqomed5ertj46p338smopp8pe1uyumg3g4enb to delete
DBG(2)[26024]> solver: ignoring package pkg(2$0$y73z87qnfy46xnsn1nr9a5ioch916y5rxdkrkbcshj5p6ci5txqb) as its state has not been changed
Checking integrity...DBG(1)[26024]> check integrity for 0 items added
 done (0 conflicting)
Your packages are up to date.
DBG(1)[26024]> release an advisory lock on a database

Local repository is mailtained by poudriere-3.1.14_2

Related part from pagesite.yaml:

{"name":"pkg","origin":"ports-mgmt/pkg","version":"1.8.7","comment":"Package manager","maintainer":"pkg@FreeBSD.org","www":"http://wiki.freebsd.org/pkgng","abi":"FreeBSD:10:amd64","arch":"freebsd:10:x86:64","prefix":"/usr/local","sum":"3bc92bbe5332106b4a74ded7c461ead9df299b504314acee3f4b6326e651dadc","flatsize":10740293,"path":"All/pkg-1.8.7.txz","repopath":"All/pkg-1.8.7.txz","licenselogic":"single","licenses":["BSD2CLAUSE"],"pkgsize":2551028,"desc":"Package management tool\n\nWWW: http://wiki.freebsd.org/pkgng","categories":["ports-mgmt"],"shlibs_provided":["libpkg.so.3"]}
Comment 1 Axel.Rau 2016-07-12 10:23:24 UTC
As one can see on the original report, both repo configs have no 'PRIORITY' attribute.
After adding priorities, the following happened (in a different jail):
- - -
[imap4:/var/tmp] root# pkg upgrade
Updating lrau-repo-py3 repository catalogue...
lrau-repo-py3 repository is up-to-date.
Updating lrau-repo-py35 repository catalogue...
lrau-repo-py35 repository is up-to-date.
Updating lrau-repo repository catalogue...
lrau-repo repository is up-to-date.
All repositories are up-to-date.
New version of pkg detected; it needs to be installed first.
The following 1 package(s) will be affected (of 0 checked):

Installed packages to be UPGRADED:
	pkg: 1.8.6 -> 1.8.7 [lrau-repo-py3]

2 MiB to be downloaded.

Proceed with this action? [y/N]: y
[imap4] Fetching pkg-1.8.7.txz: 100%    2 MiB   1.3MB/s    00:02    
Checking integrity... done (0 conflicting)
[imap4] [1/1] Upgrading pkg from 1.8.6 to 1.8.7...
[imap4] [1/1] Extracting pkg-1.8.7: 100%
You may need to manually remove /usr/local/etc/pkg.conf if it is no longer needed.
/usr/local/lib/libpkg.so.3: Undefined symbol "openat"
- - -
Because of this error, the installed pkg was unusable.
pkg-static worked as expected. The libpkg problem was not reproducible.
The complete rebuild of lrau-repo-py3 did not show any errors (log available on request).

It seems, pkg requires PRIORITY, otherwise it loops in multi-repo situations.
Comment 2 Walter Schwarzenfeld 2018-01-17 07:29:30 UTC
Is this still relevant?
Comment 3 Axel.Rau 2018-01-17 10:35:46 UTC
(In reply to w.schwarzenfeld from comment #2)
No, not really.
Sometimes things like
  /usr/local/lib/libpkg.so.4: Undefined symbol "utimensat"
happen.
I learned to always keep a pkg-static at a secure place.