Bug 211141 - ports-mgmt/pkg: pkg+pw: Doesn't run pwd_mkdb, maybe other problems
Summary: ports-mgmt/pkg: pkg+pw: Doesn't run pwd_mkdb, maybe other problems
Status: Closed FIXED
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Many People
Assignee: freebsd-pkg (Nobody)
URL:
Keywords: needs-qa
Depends on:
Blocks:
 
Reported: 2016-07-15 14:06 UTC by Nikolai Lifanov
Modified: 2018-03-20 22:46 UTC (History)
6 users (show)

See Also:
koobs: maintainer-feedback? (pkg)
koobs: merge-quarterly?


Attachments
pkg -ddd install -y mysql56-server (34.17 KB, text/plain)
2016-07-18 17:24 UTC, Nikolai Lifanov
no flags Details
pkg -ddd install -y mysql56-server (34.17 KB, text/plain)
2016-07-18 17:26 UTC, Nikolai Lifanov
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Nikolai Lifanov 2016-07-15 14:06:34 UTC
On 11.0-BETA1, pkg 1.8.7 can't create users correctly.

For example, this:
[33/36] Installing sane-backends-1.0.25...
===> Creating groups.
Creating group 'saned' with gid '194'.
===> Creating users
Creating user 'saned' with uid '194'.
pw: user 'saned' disappeared during update
pkg: PRE-INSTALL script failed

I can "fix" this by running pwd_mkdb by hand after the failure and reinstalling the affected package. Same happens with other packages that add users.
Comment 1 Nikolai Lifanov 2016-07-18 15:01:11 UTC
I'm not sure that this has been triaged correctly. I checked, and the problem is definitely not present with 10.3-RELEASE, but is present with 11.0-BETA1.
Comment 2 Kubilay Kocak freebsd_committer freebsd_triage 2016-07-18 15:26:54 UTC
The original category was base system / bin, the post-triage category is ports & packages, which (as a product) doesn't have versions except head/quarterly (hence 'latest'). This is different than the version under which a particular issue is occurring (the system with the issue)

If the issue is *actually* in the pkg(8) bootstrap system or binary in the base system, rather than the pkg port/package (the third party code that is installed by the bootstrap), then the issue should indeed be re-classified. If not, it is correctly classified currently, and further should/could also be reported upstream: https://github.com/freebsd/pkg

It's unclear at the moment why 10.3 is unaffected but 11.0 is not, but that does not itself necessarily mean its a base issue

Leaving to maintainer to classify accordingly post investigation
Comment 3 Matthew Seaman freebsd_committer freebsd_triage 2016-07-18 15:32:38 UTC
I can't reproduce this on 11.0-STABLE, admittedly with a different package
(trousers-0.3.13) that creates a new userid

Could you try checking the debug output? eg. by running:

   pkg -d -d -d -d install sane-backend

That should pretty much trace every step pkg does, including the call to pw(8).
Towards the end you should see something like this (different user and group
obviously):

{{{
DBG(3)[21154]> Scripts: executing
--- BEGIN ---
set -- trousers-0.3.13
if [ -n "${PKG_ROOTDIR}" ] && [ "${PKG_ROOTDIR}" != "/" ]; then
  PW="/usr/sbin/pw -R ${PKG_ROOTDIR}"
else
  PW=/usr/sbin/pw
fi
echo "===> Creating groups."
if ! ${PW} groupshow _tss >/dev/null 2>&1; then
  echo "Creating group '_tss' with gid '601'."
  ${PW} groupadd _tss -g 601
else
  echo "Using existing group '_tss'."
fi
echo "===> Creating users"
if ! ${PW} usershow _tss >/dev/null 2>&1; then
  echo "Creating user '_tss' with uid '601'."
  ${PW} useradd _tss -u 601 -g 601 -L daemon -c "TrouSerS user" -d /var/empty -s /usr/sbin/nologin
else
  echo "Using existing user '_tss'."
fi
Scripts: --- END ---
}}}
Comment 4 Nikolai Lifanov 2016-07-18 16:26:40 UTC
Hmm, I can reproduce this on systems upgraded from 10.3 but not clean installs.
Comment 5 Glen Barber freebsd_committer freebsd_triage 2016-07-18 16:31:45 UTC
(In reply to Nikolai Lifanov from comment #4)
> Hmm, I can reproduce this on systems upgraded from 10.3 but not clean
> installs.

Is the pkg(8) version the same on both the upgraded 10.3 and 11.0 installs?
Comment 6 Nikolai Lifanov 2016-07-18 17:22:57 UTC
Glen, pkg version is the same: 1.8.7.
Comment 7 Nikolai Lifanov 2016-07-18 17:24:19 UTC
Created attachment 172669 [details]
pkg -ddd install -y mysql56-server

I'm attaching a log of the failure.

This is on a brand-new 10.3-RELEASE install upgraded to 11.0-BETA1:

# svnlite export https://svn.freebsd.org/base/stable/11/usr.sbin/freebsd-update/freebsd-update.sh freebsd-update
# ./freebsd-update upgrade -r 11.0-BETA1
# ./freebsd-update install
<reboot>
# ./freebsd-update install
# ./freebsd-update install
Comment 8 Nikolai Lifanov 2016-07-18 17:26:14 UTC
Created attachment 172670 [details]
pkg -ddd install -y mysql56-server

oops, wrong file
Comment 9 Fukang Chen freebsd_committer freebsd_triage 2017-04-11 09:19:29 UTC
Short answer:
/etc/master.passwd contains a new user _ypldap which wasn't added into /etc/pwd.db during the upgrade to 11.0.

How to repeat:
# svnlite cat svn://svn.freebsd.org/base/release/10.3.0/etc/master.passwd > /etc/master.passwd
# pwd_mkdb -p master.passwd
# svnlite cat svn://svn.freebsd.org/base/release/11.0.1/etc/master.passwd > /etc/master.passwd

# pw useradd _tss -u 601 -g 601 -L daemon -c "TrouSerS user" -d /var/empty -s /usr/sbin/nologin
pw: user '_tss' disappeared during update

Why getpwnam(3) can't find the user '_tss' in /etc/pwd.db?

There are three copies of the original data in /etc/pwd.db,
- key pw_name field prepended by the _PW_KEYBYNAME
- key pw_uid field prepended by the _PW_KEYBYUID
- key line number in the original file prepended by the _PW_KEYBYNUM

and getpwnam(3) searches the database using the thrid key: line number
https://svnweb.freebsd.org/base/release/11.0.1/lib/libc/gen/getpwent.c?view=markup#l1842

10.3 /etc/master.passwd
23  hast:*:845:845::0:0:HAST unprivileged user:/var/empty:/usr/sbin/nologin
24  nobody:*:65534:65534::0:0:Unprivileged user:/nonexistent:/usr/sbin/nologin

10.3 /etc/pwd.db
B htonl(23) => hast
B htonl(24) => nobody

when running "pw useradd _tss -u" on 11.0, pwd_mkdb(8) inserts the
'_tss' user with the current line number as the key

11.0 /etc/master.passwd:
23  _ypldap:*:160:160::0:0:YP LDAP unprivileged user:/var/empty:/usr/sbin/nologin
24  hast:*:845:845::0:0:HAST unprivileged user:/var/empty:/usr/sbin/nologin
25  nobody:*:65534:65534::0:0:Unprivileged user:/nonexistent:/usr/sbin/nologin
26  _tss:*:601:601:daemon:0:0:TrouSerS user:/var/empty:/usr/sbin/nologin

11.0 /etc/pwd.db
B htonl(23) => hast
B htonl(24) => nobody
B htonl(26) => _tss

getpwnam(3) stopped after it tried to fetch the key 'B htonl(25)' :P
https://svnweb.freebsd.org/base/release/11.0.1/lib/libc/gen/getpwent.c?view=markup#l1858
Comment 10 Brad Davis freebsd_committer freebsd_triage 2018-03-20 22:46:55 UTC
IIRC, this was cause by bugs in pw(8) that been fixed in 11.x and is actually not related to pkg at all.

Please let me know / reopen if that is not the case.