Bug 221847

Summary: bsdconfig in FreeBSD-11.1-RELEASE-arm-armv6-RPI-B.img.xz can't install packages
Product: Base System Reporter: Tarjei Jensen <tarjei>
Component: armAssignee: freebsd-arm (Nobody) <freebsd-arm>
Status: New ---    
Severity: Affects Only Me CC: dteske, grahamperrin, linimon
Priority: ---    
Version: 11.1-RELEASE   
Hardware: arm   
OS: Any   
Attachments:
Description Flags
bsdconfig.log none

Description Tarjei Jensen 2017-08-27 08:15:46 UTC
Created attachment 185800 [details]
bsdconfig.log

Software installed on Raspberry PI B+

pkg on command line seems to work fine. 
installation of pkg worked fine and I could install kermit.

I have checked that ftp works from the machine against ftp.freebsd.org.

I have difficulty in interpreting the attached log file.
Comment 1 Mark Linimon freebsd_committer freebsd_triage 2017-08-27 12:00:07 UTC
Somewhere in the middle of all that logfile stuff I see:

pkg: Warning: Major OS version upgrade detected.  Running "pkg-static install -f pkg" recommended

Please try that and let us know the result.
Comment 2 Tarjei Jensen 2017-08-27 12:26:36 UTC
root@RPI01-03:~ # pkg-static install -f pkg
pkg-static: Warning: Major OS version upgrade detected.  Running "pkg-static install -f pkg" recommended
Updating FreeBSD repository catalogue...
FreeBSD repository is up to date.
All repositories are up to date.
Checking integrity... done (0 conflicting)
The following 1 package(s) will be affected (of 0 checked):

Installed packages to be REINSTALLED:
        pkg-1.10.1

Number of packages to be reinstalled: 1

Proceed with this action? [y/N]: y
[1/1] Reinstalling pkg-1.10.1...
[1/1] Extracting pkg-1.10.1: 100%


>>> Still the same problem. And it keeps harping about me installing pkg.

root@RPI01-03:~ # pkg list
pkg: Warning: Major OS version upgrade detected.  Running "pkg-static install -f pkg" recommended
/usr/local/bin/bash
/usr/local/bin/bashbug
/usr/local/bin/rbash
/usr/local/include/bash/alias.h
Comment 3 Tarjei Jensen 2017-08-27 16:36:47 UTC
I've had a brainwave. There is two pkg binaries. One for 11.0 and one for 11.1

root@RPI01-03:~ # file /usr/local/sbin/pkg
/usr/local/sbin/pkg: ELF 32-bit LSB executable, ARM, EABI5 version 1 (FreeBSD), dynamically linked, interpreter /libexec/ld-elf.so.1, for FreeBSD 11.0 (1100122), FreeBSD-style, stripped

root@RPI01-03:~ # file /usr/sbin/pkg
/usr/sbin/pkg: ELF 32-bit LSB executable, ARM, EABI5 version 1 (FreeBSD), dynamically linked, interpreter /libexec/ld-elf.so.1, for FreeBSD 11.1, FreeBSD-style, stripped

Copying /usr/sbin/pkg to /usr/local/sbin/pkg caused that specific issue to go away. But finding packages takes hours. I suspect that is not what should happen.

I suspect that just copying the pkg file to the correct place is not enough to fix the issue.


ps -auxfw


USER     PID %CPU %MEM   VSZ  RSS TT  STAT STARTED      TIME COMMAND
root    1467 97.7  1.0  5936 4952  0  R+   15:16   191:04.16 pkg -N -vv
root       0  0.0  0.0     0   32  -  DLs  13:59     0:00.16 [kernel]
root       1  0.0  0.2  5260  868  -  ILs  13:59     0:00.05 /sbin/init --
root       2  0.0  0.0     0    8  -  DL   13:59     0:00.00 [crypto]
root       3  0.0  0.0     0    8  -  DL   13:59     0:00.00 [crypto returns]
root       4  0.0  0.0     0   16  -  DL   13:59     0:00.00 [cam]
root       5  0.0  0.0     0    8  -  IL   13:59     0:00.00 [VCHIQ-0]
root       6  0.0  0.0     0    8  -  IL   13:59     0:00.00 [VCHIQr-0]
root       7  0.0  0.0     0    8  -  IL   13:59     0:00.00 [VCHIQs-0]
root       8  0.0  0.0     0    8  -  DL   13:59     0:00.00 [sctp_iterator]
root       9  0.0  0.0     0    8  -  DL   13:59     0:09.41 [rand_harvestq]
root      10  0.0  0.0     0    8  -  RL   13:59    59:33.68 [idle]
root      11  0.0  0.0     0  152  -  WL   13:59     1:33.92 [intr]
root      12  0.0  0.0     0   24  -  DL   13:59     0:01.62 [geom]
root      13  0.0  0.0     0   48  -  DL   13:59     0:36.91 [usb]
root      14  0.0  0.0     0    8  -  DL   13:59     0:00.01 [soaiod1]
root      15  0.0  0.0     0    8  -  DL   13:59     0:00.01 [soaiod2]
root      16  0.0  0.0     0    8  -  DL   13:59     0:00.01 [soaiod3]
root      17  0.0  0.0     0    8  -  DL   13:59     0:00.01 [soaiod4]
root      18  0.0  0.0     0    8  -  DL   13:59     0:01.87 [mmcsd0: mmc/sd card]
root      19  0.0  0.0     0    8  -  IL   13:59     0:00.00 [VCHIQka-0]
root      20  0.0  0.0     0    8  -  IL   13:59     0:00.00 [bcm2835_audio_worke]
root      21  0.0  0.0     0   24  -  DL   13:59     0:01.04 [pagedaemon]
root      22  0.0  0.0     0    8  -  DL   13:59     0:00.00 [vmdaemon]
root      23  0.0  0.0     0    8  -  DL   13:59     0:00.00 [pagezero]
root      24  0.0  0.0     0    8  -  DL   13:59     0:00.34 [bufspacedaemon]
root      25  0.0  0.0     0   16  -  DL   13:59     0:01.47 [bufdaemon]
root      26  0.0  0.0     0    8  -  DL   13:59     0:00.31 [vnlru]
root      27  0.0  0.0     0    8  -  DL   13:59     0:04.12 [syncer]
root      28  0.0  0.0     0    8  -  DL   13:59     0:03.08 [schedcpu]
root     121  0.0  0.4  5984 1744  -  Is   13:59     0:00.01 adjkerntz -i
root     331  0.0  0.4  6252 2104  -  Is   14:00     0:00.02 dhclient: ue0 [priv] (dhclient)
_dhcp    377  0.0  0.5  6252 2208  -  Is   14:00     0:00.00 dhclient: ue0 (dhclient)
root     382  0.0  0.4  6276 1912  -  Is   14:00     0:00.01 /usr/sbin/moused -p /dev/ums0 -t auto -I /var/run/moused.ums0.pid
root     395  0.0  0.8  7352 3848  -  Is   14:00     0:00.04 /sbin/devd
root     468  0.0  0.4  6136 2112  -  Ss   14:00     0:00.26 /usr/sbin/syslogd -s
root     629  0.0  0.4  6104 1784  -  Ss   14:00     0:08.25 /usr/sbin/powerd
root     648  0.0  0.0     0    8  -  DL   14:00     0:00.06 [md99]
root     667  0.0  1.2 11728 6012  -  Is   14:00     0:00.04 /usr/sbin/sshd
root     671  0.0  0.4  6204 2152  -  Is   14:00     0:00.30 /usr/sbin/cron -s
root     966  0.0  1.4 14940 6704  -  Is   14:43     0:00.39 sshd: freebsd [priv] (sshd)
freebsd  969  0.0  1.4 14940 6780  -  I    14:43     0:01.59 sshd: freebsd@pts/0 (sshd)
root    1671  0.0  1.4 14940 6704  -  Is   16:41     0:00.28 sshd: freebsd [priv] (sshd)
freebsd 1674  0.0  1.4 14940 6788  -  S    16:42     0:04.66 sshd: freebsd@pts/1 (sshd)
root    2320  0.0  1.9  9176 9064  -  Ss   17:35     0:01.18 /usr/sbin/ntpd -c /etc/ntp.conf -p /var/run/ntpd.pid -f /var/db/ntpd.d
root     730  0.0  0.4  6124 1840 u0  Is+  14:00     0:00.02 /usr/libexec/getty 3wire ttyu0
root     729  0.0  0.5  7124 2628 v0  Is   14:00     0:00.26 login [pam] (login)
freebsd  731  0.0  0.7  7128 3180 v0  I    14:00     0:00.15 -csh (csh)
root    1335  0.0  0.5  7108 2616 v0  I    15:04     0:00.18 su - root
root    1336  0.0  0.7  7128 3156 v0  I+   15:04     0:00.25 -su (csh)
freebsd  970  0.0  0.7  7128 3232  0  Is   14:43     0:00.13 -csh (csh)
root     973  0.0  0.5  7108 2616  0  I    14:43     0:00.21 su - root
root     974  0.0  0.7  7128 3208  0  I    14:43     0:00.52 -su (csh)
root    1369  0.0  0.6  6688 2772  0  I+   15:16     0:00.42 /bin/sh /usr/sbin/bsdconfig
root    1390  0.0  0.7  6688 3380  0  I+   15:16     0:03.52 /bin/sh /usr/libexec/bsdconfig/030.packages/packages
root    1466  0.0  0.7  6688 3380  0  I+   15:16     0:00.01 /bin/sh /usr/libexec/bsdconfig/030.packages/packages
root    1468  0.0  0.4  6332 2148  0  I+   15:16     0:00.02 awk  # BEGIN-AWK\n\011\011$1 ~ /^ABI/ && $0 = $NF, sub(/^"/, "") && su
freebsd 1675  0.0  0.7  7128 3232  1  Is   16:42     0:00.12 -csh (csh)
root    1679  0.0  0.5  7108 2616  1  I    16:42     0:00.18 su - root
root    1680  0.0  0.7  7128 3224  1  S    16:42     0:00.80 -su (csh)
root    2412  0.0  0.5  6668 2292  1  R+   18:30     0:00.02 ps -auxfw
Comment 4 Tarjei Jensen 2017-08-27 17:56:52 UTC
I think I have identified why the problem occurs.

The /usr/sbin/pkg program that is included with 11.1-RELEASE installs the 11 version of pkg in /usr/local/sbin and from then on it is downhill.
Comment 5 Tarjei Jensen 2017-08-27 18:45:24 UTC
/etc/pkg/FreeBSD.conf does not point to the 11.1-Release

I'm uncertain what the url should be.

FreeBSD.conf contains the following :

# $FreeBSD: releng/11.1/etc/pkg/FreeBSD.conf 320745 2017-07-06 17:22:33Z gjb $
#
# To disable this repository, instead of modifying or removing this file,
# create a /usr/local/etc/pkg/repos/FreeBSD.conf file:
#
#   mkdir -p /usr/local/etc/pkg/repos
#   echo "FreeBSD: { enabled: no }" > /usr/local/etc/pkg/repos/FreeBSD.conf
#

FreeBSD: {
  url: "pkg+http://pkg.FreeBSD.org/${ABI}/quarterly",
  mirror_type: "srv",
  signature_type: "fingerprints",
  fingerprints: "/usr/share/keys/pkg",
  enabled: yes
}
Comment 6 Tarjei Jensen 2017-08-28 12:23:58 UTC
I investigated /usr/sbin/pkg

root@RPI01-03:/usr/sbin # strings pkg | grep -i freebsd.org
pkg+http://pkg.FreeBSD.org/${ABI}/latest


It looks like it downloads the 11 version of the pkg software and not the 11.1.

So far unlucky in determining where ABI is defined so that I can find out why ABI is not correct.

So far it is clear that the installed pkg stub is not particularly helpful.

Change in /etc/pkg/FreeBSD.conf
   url: "pkg+http://pkg.FreeBSD.org/${ABI}/quarterly",

                  to

   url: "pkg+http://pkg.freebsd.org/${ABI}/latest",

did not help. Still pulled version 11 pkg and failed fast in bsdconfig. It would have helped if the log file had more information on what bsdconfig tries to do. Including the URLs it uses.
Comment 7 Tarjei Jensen 2017-08-28 18:13:35 UTC
I have updated /etc/pkg/FreeBSD.conf to use

I have updated pkg with an v 11 binary since that appears to be what's there.

I've run bsdconfig with logging. And tried to download packages using HTTP.

Here is what the log says about access to files. It is simply unable to find the packages on the servers. The structure on the server and what the pkg program looks for is totally different.

DEBUG: f_getvar: var=[_httpPath] value=[http://pkg.freebsd.org] r=0
DEBUG: Looks like a DNS name or IPv4 addr: pkg.freebsd.org
DEBUG: hostname = `pkg.freebsd.org'
DEBUG: f_media_set_http: Looking up hostname, pkg.freebsd.org, using host(1)
DEBUG: f_host_lookup: host=[pkg.freebsd.org]
DEBUG: f_getvar: var=[_httpPath] value=[http://pkg.freebsd.org] r=0
DEBUG: Found DNS entry for pkg.freebsd.org successfully.
DEBUG: f_getvar: var=[_struct_value_device_http_name] value=[http://pkg.freebsd.org] r=0
DEBUG: f_getvar: var=[httpHost] value=[pkg.freebsd.org] r=0
DEBUG: f_http_check_access: Looking up hostname, pkg.freebsd.org, using host(1)
DEBUG: f_host_lookup: host=[pkg.freebsd.org]
DEBUG: f_getvar: var=[_httpPath] value=[http://pkg.freebsd.org] r=0
DEBUG: f_getvar: var=[_httpPath] value=[http://pkg.freebsd.org] r=0
DEBUG: f_getvar: var=[_httpPath] value=[http://ftp.freebsd.org] r=0
DEBUG: Looks like a DNS name or IPv4 addr: ftp.freebsd.org
DEBUG: hostname = `ftp.freebsd.org'
DEBUG: f_media_set_http: Looking up hostname, ftp.freebsd.org, using host(1)
DEBUG: f_host_lookup: host=[ftp.freebsd.org]
DEBUG: f_getvar: var=[_httpPath] value=[http://ftp.freebsd.org] r=0
DEBUG: Found DNS entry for ftp.freebsd.org successfully.
DEBUG: f_getvar: var=[_struct_value_device_http_name] value=[http://ftp.freebsd.org] r=0
DEBUG: f_getvar: var=[httpHost] value=[ftp.freebsd.org] r=0
DEBUG: f_http_check_access: Looking up hostname, ftp.freebsd.org, using host(1)
DEBUG: f_host_lookup: host=[ftp.freebsd.org]
DEBUG: f_getvar: var=[_httpPath] value=[http://ftp.freebsd.org] r=0
DEBUG: f_getvar: var=[httpHost] value=[ftp.freebsd.org] r=0
DEBUG: f_getvar: var=[_httpPath%/] value=[http://ftp.freebsd.org] r=0
DEBUG: f_getvar: var=[httpHost] value=[ftp.freebsd.org] r=0
DEBUG: f_http_check_access: Looking up hostname, ftp.freebsd.org, using host(1)
DEBUG: f_host_lookup: host=[ftp.freebsd.org]
DEBUG: f_getvar: var=[_httpPath] value=[http://ftp.freebsd.org/FreeBSD:11:armv6/latest] r=0
DEBUG: f_getvar: var=[_httpPath] value=[http://ftp.freebsd.org/FreeBSD:11:armv6/latest] r=0
DEBUG: f_getvar: var=[httpHost] value=[ftp.freebsd.org] r=0
DEBUG: f_http_check_access: Looking up hostname, ftp.freebsd.org, using host(1)
DEBUG: f_host_lookup: host=[ftp.freebsd.org]
DEBUG: f_getvar: var=[_httpPath] value=[http://ftp.freebsd.org/./11.1-RELEASE] r=0
DEBUG: f_getvar: var=[_httpPath] value=[http://ftp.freebsd.org/./11.1-RELEASE] r=0
DEBUG: f_getvar: var=[httpHost] value=[ftp.freebsd.org] r=0
DEBUG: f_http_check_access: Looking up hostname, ftp.freebsd.org, using host(1)
DEBUG: f_host_lookup: host=[ftp.freebsd.org]
DEBUG: f_getvar: var=[_httpPath] value=[http://ftp.freebsd.org/releases/armv6/11.1-RELEASE] r=0
DEBUG: f_getvar: var=[_httpPath] value=[http://ftp.freebsd.org/releases/armv6/11.1-RELEASE] r=0
DEBUG: f_getvar: var=[httpHost] value=[ftp.freebsd.org] r=0
DEBUG: f_http_check_access: Looking up hostname, ftp.freebsd.org, using host(1)
DEBUG: f_host_lookup: host=[ftp.freebsd.org]
DEBUG: f_getvar: var=[_httpPath] value=[http://ftp.freebsd.org/snapshots/armv6/11.1-RELEASE] r=0
DEBUG: f_getvar: var=[_httpPath] value=[http://ftp.freebsd.org/snapshots/armv6/11.1-RELEASE] r=0
DEBUG: f_getvar: var=[httpHost] value=[ftp.freebsd.org] r=0
DEBUG: f_http_check_access: Looking up hostname, ftp.freebsd.org, using host(1)
DEBUG: f_host_lookup: host=[ftp.freebsd.org]
DEBUG: f_getvar: var=[_httpPath] value=[http://ftp.freebsd.org/pub/FreeBSD/11.1-RELEASE] r=0
DEBUG: f_getvar: var=[_httpPath] value=[http://ftp.freebsd.org/pub/FreeBSD/11.1-RELEASE] r=0
DEBUG: f_getvar: var=[httpHost] value=[ftp.freebsd.org] r=0
DEBUG: f_http_check_access: Looking up hostname, ftp.freebsd.org, using host(1)
DEBUG: f_host_lookup: host=[ftp.freebsd.org]
DEBUG: f_getvar: var=[_httpPath] value=[http://ftp.freebsd.org/pub/FreeBSD/releases/armv6/11.1-RELEASE] r=0
DEBUG: f_getvar: var=[_httpPath] value=[http://ftp.freebsd.org/pub/FreeBSD/releases/armv6/11.1-RELEASE] r=0
DEBUG: f_getvar: var=[httpHost] value=[ftp.freebsd.org] r=0
DEBUG: f_http_check_access: Looking up hostname, ftp.freebsd.org, using host(1)
DEBUG: f_host_lookup: host=[ftp.freebsd.org]
DEBUG: f_getvar: var=[_httpPath] value=[http://ftp.freebsd.org/pub/FreeBSD/snapshots/armv6/11.1-RELEASE] r=0
DEBUG: f_getvar: var=[_httpPath] value=[http://ftp.freebsd.org/pub/FreeBSD/snapshots/armv6/11.1-RELEASE] r=0
DEBUG: f_getvar: var=[httpHost] value=[ftp.freebsd.org] r=0
DEBUG: f_http_check_access: Looking up hostname, ftp.freebsd.org, using host(1)
DEBUG: f_host_lookup: host=[ftp.freebsd.org]
DEBUG: f_getvar: var=[_httpPath] value=[http://ftp.freebsd.org/pub/FreeBSD-Archive/old-releases/armv6/11.1-RELEASE] r=0
DEBUG: f_getvar: var=[_httpPath] value=[http://ftp.freebsd.org/pub/FreeBSD-Archive/old-releases/armv6/11.1-RELEASE] r=0
DEBUG: f_getvar: var=[_httpPath] value=[http://ftp.freebsd.org/pub/FreeBSD-Archive/old-releases/armv6/11.1-RELEASE] r=0
Comment 8 Tarjei Jensen 2017-08-29 08:50:57 UTC
I have compiled pkg in ports and forced installation. That helped a lot!

I started bsdconfig -> Packages and I used the "HTTP Direct" option.
   "dist Main Site" did not work.
   "pkg Main Site" worked.

As of now bsdconfig Packages "works" on my RPI B+. I use the term "works" because it is pretty clear that nobody gives a damn about whether it works or is useful. Which is probably why FreeBSD has not made a larger impact.


Root cause of problem : system not tested and nobody cares.
Comment 9 Mark Linimon freebsd_committer freebsd_triage 2017-08-29 08:55:40 UTC
I'm sorry that I have not given more attention to your PR.  However:

 - there are dozens that come in every day.
 - I have a hard deadline that I am trying to meet by the end of the week.
 - comments such as the ones you just made are not helpful.
Comment 10 Mark Linimon freebsd_committer freebsd_triage 2017-08-29 08:58:10 UTC
(In reply to Tarjei Jensen from comment #4)

This is by design.

Packages are built on the "earliest supported branch of a release".  This allows for maximum coverage.  e.g. packages built for 11.0 are supposed to work on 11.1.

The reverse is _not_ the case: packages built on 11.1 are _not_ guaranteed to work on 11.0.  They might ... but it depends on what has changed in the src tree between 11.0 and 11.1.

tl;dr: forward compatibility is supposed to work.  If it doesn't, it's a bug.  Backwards compatibility might or might not work.
Comment 11 Tarjei Jensen 2017-08-29 09:23:10 UTC
Even the most cursory testing would have indicated that something was not right.

So it is pretty clear that nobody cares. Because if they did. They would have checked if the basic configuration can do its job.
Comment 12 Devin Teske freebsd_committer freebsd_triage 2018-06-17 22:17:36 UTC
And if there is anything I can do to make bsdconfig better, let me know.

Like, for example, if there are options that don't work in the menu that we should remove.