Bug 261755

Summary: OpenSSH: Merge 8.8 to stable/13: 7.9 does not work with hardware (Yubico) security keys
Product: Base System Reporter: Ian <iandstanley>
Component: binAssignee: Ed Maste <emaste>
Status: Closed FIXED    
Severity: Affects Some People CC: chris, emaste, herbert, iandstanley, zarychtam
Priority: --- Keywords: needs-qa
Version: 13.0-RELEASEFlags: koobs: mfc-stable13?
koobs: mfc-stable12-
Hardware: Any   
OS: Any   
URL: https://www.openssh.com/releasenotes.html

Description Ian 2022-02-06 19:59:27 UTC
On a recent install of FreeBSD 13.0 RELEASE (and updated with freebsd-update) I was having issues with my Yubico security key and ssh when I noticed that the version of SSH shipped with 13.0 is version 7.9 (2018 vintage)

$ ssh -V 
OpenSSh_7.9p1, OpenSSL 1.1.1k-freebsd 24 Aug 2021

$ which ssh
/usr/bin/ssh

Version 7.9 was released in 2018 and I was shocked that all we did was just recompile a 3 year old version of a commonly used security tool that has had a series of security fixes since and was out of date in April 2019. Version 7.9 has at least 10 CVEs attributed to it that have been fixed in the 8.8 version found in ports.

But we all assume that when a new release appears critical tools get updated to at least the current version at the time of fixing the release branch.

This missed update ought to have appeared in 2020 in 11.4 or at least in 12.0, not unresolved in v13.0 in 2022. 

If I hadn't been trying to use a new feature of Openssh 8.2 I wouldn't have noticed that I was using a version 3 years out of date. 


BACKGROUND
I had been trying to run ssh-add -K to add the resident key form the yubikey

After I realized that it was a old version I installed the openssh-portable version 8.8 from the repository

$ /usr/local/bin/ssh -V 
OpenSSh_8.8p1, OpenSSL 1.1.1k-freebsd 24 Aug 2021

Compared with:

$ /usr/bin/ssh -V 
OpenSSh_7.9p1, OpenSSL 1.1.1k-freebsd 24 Aug 2021

I was surprised that the latest release had not upgraded a critical security tool to at least 8.2 (or later) which was released TWO YEARS ago particularly seeing that there are at 10 vulnerabilities between the version in /usr/bin/ssh and openssh-portable. 

It would also mean that I would not need to patch a bunch of scripts and setup aliases.


VULNERABILITIES:

https://www.cvedetails.com/vulnerability-list/vendor_id-97/product_id-585/Openbsd-Openssh.html
Comment 1 Ian 2022-02-06 20:02:04 UTC
Proposal: update to 8.2 or later (preferably the 8.8 from ports which is the latest version with a whole load of patches/fixes/updates
Comment 2 Herbert J. Skuhra 2022-02-06 20:52:27 UTC
OpenSSH 8.8p1 is already in HEAD.

commit e9e8876a4d6afc1ad5315faaa191b25121a813d7
Merge: 71a1539e378 4f19900354c
Author: Ed Maste
Date:   Sun Dec 19 11:02:02 2021 -0500

    ssh: update to OpenSSH v8.8p1

$ uname -rms
FreeBSD 14.0-CURRENT amd64
$ /usr/bin/ssh -V
OpenSSH_8.8p1, OpenSSL 1.1.1m-freebsd  14 Dec 2021

I have no idea what blocks the MFC.
Comment 3 Ed Maste freebsd_committer freebsd_triage 2022-02-07 21:58:20 UTC
I will MFC 8.8 soon, certainly before FreeBSD 13.1.

Note that security updates have been applied to 7.9p1 in 12/13.
Comment 4 Marek Zarychta 2022-02-07 22:00:35 UTC
Much of the legwork is already being done but needs more of our appreciation. 
Ed: thanks for 2FA ssh working in 14-CURRENT !
Comment 5 Ed Maste freebsd_committer freebsd_triage 2022-02-07 23:58:16 UTC
MFC for testing available at https://github.com/emaste/freebsd/commit/c7d1ed4105bb91af7d29b5d78668c2e378b0e629

Downloadable as a patch from https://github.com/emaste/freebsd/commit/c7d1ed4105bb91af7d29b5d78668c2e378b0e629.patch (too large to attach to this PR)

Much obliged if you can apply and test
Comment 6 Herbert J. Skuhra 2022-02-08 08:41:37 UTC
Is this the correct patch? 'ssh -V' shows 8.7p1 not 8.8p1.
I've tested the patch on two systems. It applies with some whitespace warnings but otherwise all is OK so far. 

/home/herbert/c7d1ed4105bb91af7d29b5d78668c2e378b0e629.patch:1483: trailing whitespace.
if [ "x" != "x$PACKAGES" ]; then 
/home/herbert/c7d1ed4105bb91af7d29b5d78668c2e378b0e629.patch:1905: trailing whitespace.
    
/home/herbert/c7d1ed4105bb91af7d29b5d78668c2e378b0e629.patch:1919: trailing whitespace.
    
/home/herbert/c7d1ed4105bb91af7d29b5d78668c2e378b0e629.patch:1957: trailing whitespace.
    
/home/herbert/c7d1ed4105bb91af7d29b5d78668c2e378b0e629.patch:2576: trailing whitespace.
    
warning: squelched 922 whitespace errors
warning: 927 lines add whitespace errors.

Should we run some special tests?

Thanks.
Comment 7 Marek Zarychta 2022-02-08 08:51:19 UTC
Patience is required here. Ed will do MFC of OpenSSH 8.8p1 to stable/13 and not only ssh will be updated but also libs required to have 2FA ssh with FIDO2 support will be added. It's going to be a set of MFCs which has to be precisely prepared, so it takes some effort.
Comment 8 Kubilay Kocak freebsd_committer freebsd_triage 2022-02-08 22:17:03 UTC
@Ed Please ref this issue in any upcoming commit (or merge) log messages, thanks!
Comment 9 Ed Maste freebsd_committer freebsd_triage 2022-02-09 04:10:57 UTC
(In reply to Herbert J. Skuhra from comment #6)
> Is this the correct patch? 'ssh -V' shows 8.7p1 not 8.8p1.

Yes, this is correct for this patch. The first MFC is a large step from 7.9p1 to 8.7p1. libfido2 & libcbor will follow, then enabling U2F support in ssh, then finally the (comparatively tiny) update from 8.7p1 to 8.8p1
Comment 10 Ed Maste freebsd_committer freebsd_triage 2022-02-09 04:15:27 UTC
> Should we run some special tests?

Nothing specific, just confirm that it behaves as you expect in your environment.

If you look at the "cherry picked from commit" lines in the MFC patch you'll see iterative fixes that were applied in head as issues were found that will offer some insight into the sort of issues encountered. pam_ssh in particular is notable.
Comment 11 Ed Maste freebsd_committer freebsd_triage 2022-02-09 17:51:23 UTC
(In reply to Herbert J. Skuhra from comment #6)
> It applies with some whitespace warnings but otherwise all is OK so far. 

This is (unfortunately) expected - it is just complaining as it applies upstream's bogus whitespace.
Comment 12 Ed Maste freebsd_committer freebsd_triage 2022-02-09 23:57:09 UTC
8.7p1 merged in:

commit 317a38ab65334cbd24bd020b20b11041423d142f
Author: Ed Maste <emaste@FreeBSD.org>
Date:   Tue Sep 7 21:05:51 2021 -0400

    openssh: update to OpenSSH v8.7p1
    
    Some notable changes, from upstream's release notes:
    
    - sshd(8): Remove support for obsolete "host/port" syntax.
    - ssh(1): When prompting whether to record a new host key, accept the key
      fingerprint as a synonym for "yes".
    - ssh-keygen(1): when acting as a CA and signing certificates with an RSA
      key, default to using the rsa-sha2-512 signature algorithm.
    - ssh(1), sshd(8), ssh-keygen(1): this release removes the "ssh-rsa"
      (RSA/SHA1) algorithm from those accepted for certificate signatures.
    - ssh-sk-helper(8): this is a new binary. It is used by the FIDO/U2F
      support to provide address-space isolation for token middleware
      libraries (including the internal one).
    - ssh(1): this release enables UpdateHostkeys by default subject to some
      conservative preconditions.
    - scp(1): this release changes the behaviour of remote to remote copies
      (e.g. "scp host-a:/path host-b:") to transfer through the local host
      by default.
    - scp(1): experimental support for transfers using the SFTP protocol as
      a replacement for the venerable SCP/RCP protocol that it has
      traditionally used.
    
    Additional integration work is needed to support FIDO/U2F in the base
    system.
    
    Deprecation Notice
    ------------------
    
    OpenSSH will disable the ssh-rsa signature scheme by default in the
    next release.
    
    Reviewed by:    imp
    MFC after:      1 month
    Relnotes:       Yes
    Sponsored by:   The FreeBSD Foundation
    Differential Revision:  https://reviews.freebsd.org/D29985
    
    (cherry picked from commit 19261079b74319502c6ffa1249920079f0f69a72)
    (cherry picked from commit f448c3ed4ae1281861913a56377f9d93d49f8e8e)
    (cherry picked from commit 1f290c707a19d1695c303e6c8ead9cc414ccc6dc)
    (cherry picked from commit 0f9bafdfc325779e4ecc5154d5bb06c752297138)
    (cherry picked from commit adb56e58e8db84d8087ebe3d3e7def0074cb5a90)
    (cherry picked from commit 576b58108c1723c85e4dd00355e29bfe301dab11)
    (cherry picked from commit 1c99af1ebe61cbaf633792941640dcd254acf921)
    (cherry picked from commit 87152f34054921632016bc5eb4ab9f836fbaa522)
    (cherry picked from commit 172fa4aa7577915bf5ace5783251821d3774dc05)
Comment 13 Ed Maste freebsd_committer freebsd_triage 2022-02-11 20:24:16 UTC
U2F enabled in:

commit a613d68fff9af03730e1c18438f85d80649547e4
Author: Ed Maste <emaste@FreeBSD.org>
Date:   Wed Oct 6 23:31:17 2021 -0400

    ssh: enable FIDO/U2F keys
    
    Description of FIDO/U2F support (from OpenSSH 8.2 release notes,
    https://www.openssh.com/txt/release-8.2):
    
      This release adds support for FIDO/U2F hardware authenticators to
      OpenSSH. U2F/FIDO are open standards for inexpensive two-factor
      authentication hardware that are widely used for website
      authentication.  In OpenSSH FIDO devices are supported by new public
      key types "ecdsa-sk" and "ed25519-sk", along with corresponding
      certificate types.
    
      ssh-keygen(1) may be used to generate a FIDO token-backed key, after
      which they may be used much like any other key type supported by
      OpenSSH, so long as the hardware token is attached when the keys are
      used. FIDO tokens also generally require the user explicitly
      authorise operations by touching or tapping them.
    
      Generating a FIDO key requires the token be attached, and will
      usually require the user tap the token to confirm the operation:
    
        $ ssh-keygen -t ecdsa-sk -f ~/.ssh/id_ecdsa_sk
        Generating public/private ecdsa-sk key pair.
        You may need to touch your security key to authorize key generation.
        Enter file in which to save the key (/home/djm/.ssh/id_ecdsa_sk):
        Enter passphrase (empty for no passphrase):
        Enter same passphrase again:
        Your identification has been saved in /home/djm/.ssh/id_ecdsa_sk
        Your public key has been saved in /home/djm/.ssh/id_ecdsa_sk.pub
    
      This will yield a public and private key-pair. The private key file
      should be useless to an attacker who does not have access to the
      physical token. After generation, this key may be used like any
      other supported key in OpenSSH and may be listed in authorized_keys,
      added to ssh-agent(1), etc. The only additional stipulation is that
      the FIDO token that the key belongs to must be attached when the key
      is used.
    
    To enable FIDO/U2F support, this change regenerates ssh_namespace.h,
    adds ssh-sk-helper, and sets ENABLE_SK_INTERNAL (unless building
    WITHOUT_USB).
    
    devd integration is not included in this change, and is under
    investigation for the base system.  In the interim the security/u2f-devd
    port can be installed to provide appropriate devd rules.
    
    Reviewed by:    delphij, kevans
    Relnotes:       Yes
    Sponsored by:   The FreeBSD Foundation
    Differential Revision:  https://reviews.freebsd.org/D32509
    
    (cherry picked from commit e9a994639b2af232f994ba2ad23ca45a17718d2b)

8.8 merged in:

commit 8464ad72e0874fb70c5eb96fe14225c18d06fb3a
Author: Ed Maste <emaste@FreeBSD.org>
Date:   Sun Dec 19 11:02:02 2021 -0500

    ssh: update to OpenSSH v8.8p1
    
    OpenSSH v8.8p1 was motivated primarily by a security update and
    deprecation of RSA/SHA1 signatures.  It also has a few minor bug fixes.
    
    The security update was already applied to FreeBSD as an independent
    change, and the RSA/SHA1 deprecation is excluded from this commit but
    will immediately follow.
    
    MFC after:      1 month
    Relnotes:       Yes
    Sponsored by:   The FreeBSD Foundation
    
    (cherry picked from commit e9e8876a4d6afc1ad5315faaa191b25121a813d7)
    (cherry picked from commit 2ffb13149c8e46cb7d7e891b237255615906dc60)

stable/12 in progress