Bug 221393

Summary: Logins stopped working?
Product: Services Reporter: Mikhail T. <freebsd-2024>
Component: Core InfrastructureAssignee: Cluster Admin <clusteradm>
Status: Closed FIXED    
Severity: Affects Only Me CC: accounts, david, peter, sbruno
Priority: ---    
Version: unspecified   
Hardware: Any   
OS: Any   

Description Mikhail T. 2017-08-10 15:59:41 UTC
The ssh keys I've been using for years stopped working this morning...

I can not update my svn repos (much less commit anything) and ssh-ing into freefall is not working either:

...
debug1: Server host key: ssh-rsa 
SHA256:S0kycszN0jTAKt6Pr9TncuWjx3U2S2vK7uepixbkqwQ
debug1: found 8 insecure fingerprints in DNS
debug1: matching host key fingerprint found in DNS
debug1: Host 'freefall.freebsd.org' is known and matches the RSA host key.
debug1: Found key in /home/mi/.ssh/known_hosts:213
debug1: rekey after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: rekey after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS received
debug1: SSH2_MSG_EXT_INFO received
debug1: Fssh_kex_input_ext_info: server-sig-algs=<ssh-ed25519,ssh-rsa,ssh-dss,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521>
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey
debug1: Next authentication method: publickey
debug1: Offering RSA public key: /home/mi/.ssh/id_rsa
debug1: Authentications that can continue: publickey
debug1: Offering DSA public key: /home/mi/.ssh/id_dsa
debug1: Authentications that can continue: publickey
debug1: No more authentication methods to try.
Permission denied (publickey).

Please, advise. Thanks!

(In addition, I note that finger USER@FreeBSD.org no longer works either -- getting "connection refused"...)
Comment 1 Glen Barber freebsd_committer freebsd_triage 2017-08-10 16:27:07 UTC
Please see the email from Peter sent to the developer's list.
Message-ID: <1960380.yq5XZkDxI2@overcee.wemm.org>

For finger(1), that is only accessible from inside the cluster.
Comment 2 Mikhail T. 2017-08-10 16:28:35 UTC
(In reply to Glen Barber from comment #1)
> For finger(1), that is only accessible from inside the cluster.

That's a new limitation, it used to be possible -- and not too long ago...
Comment 3 Glen Barber freebsd_committer freebsd_triage 2017-08-10 16:31:56 UTC
(In reply to Mikhail T. from comment #2)
> (In reply to Glen Barber from comment #1)
> > For finger(1), that is only accessible from inside the cluster.
> 
> That's a new limitation, it used to be possible -- and not too long ago...

Correct.  It was done a few months ago.
Comment 4 Mikhail T. 2017-08-10 16:37:26 UTC
(In reply to Glen Barber from comment #3)
> It [disabling finger from outside of the cluster] was done a few months ago.
Not to distract this ticket away from its main purpose, but I consider the change to finger unwelcome and detrimental. If it was extensively discussed by project-members already, then so be it. Otherwise, if it was a decision by a smaller group of people, I'd ask for it to be reverted... There is no data exposed by finger, that's not also available via other means (such as mailing lists, lists of project-contributors).
Comment 5 Mikhail T. 2017-08-11 18:46:36 UTC
Ping? Still can't do anything -- and I was going to catch up on my PRs this weekend...
Comment 6 Mathieu Arnold freebsd_committer freebsd_triage 2017-08-11 20:17:39 UTC
It is like gjb said the first comment:

Please see the email from Peter sent to the developer's list.
Message-ID: <1960380.yq5XZkDxI2@overcee.wemm.org>
Comment 7 Mikhail T. 2017-08-11 20:33:05 UTC
I'm afraid, I can not find that message anywhere... Could it be attached to this ticket or forwarded to me by e-mail? Thanks!
Comment 8 Mikhail T. 2017-10-03 04:02:08 UTC
Presumably, the problem is with the type(s) of my keys. Could somebody, please, replace my current key-collection with:

ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA7AYehimrCIeAnXfuH1nF4fyVzPqJiddokn2SG90mZDncfA2n0is+hOtJpHZdIGoehg+4SHl5A8ksZ6lVyU1KPt/Q/r8abLeMxLJbXWwVkdPltcMfP1m44j/BGE+qP4wXNj8jVvAJzSKj5t2nfkkR7XA3kfLpijvQgkUgvXCAmXuaug9MBksSWB+fc351XN4m0gRj0hcLk0roqkSS3FqkzYAGHqsFsSQqHUeYhPd4o8qgkLMcmAxqjjzUOYNoB8lNt6zI6yOjX4hqOBRWUwcwzC0Fk+hV6s7DFGkIHccdgs3l4C3z+g8XTTJ5Gcq6cu4I26vStJcRIgvXwqrmS2Nr8Q== mi@symbion
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQZ5PDl+8YZydxylF5ntPA9YJ7JDCjfyCe7dpdSkDtFZZHRTcw9MJRWy7eccNrbhsZ0ULB44Jb7rY0RL45bo5VKQb3M2mCalIZVcuUNwR7DQtlsPYC5eS0Vhjv0F99Mu7UFhokYzpx1qyZ/b6592U2DCdplFOA5BFphbP783NVvdVPUcKYyJYiSRH4vnaEc2n2QOOyhbGiCQQOas1sEg6vJTBE3WMbq9o9E+3Ja+emtsFSCAoM/roR4iIAFKtc9iYZ9c5LKwhZHS+EtK45NtOIuqdSb6/STEDzNTat1KT/OC/Qr45QFPHQEX17tKCGNbKYo9RNRsUG0QICwG5xAqOBDx mi@narawntapu

Thank you!
Comment 9 Sean Bruno freebsd_committer freebsd_triage 2017-11-05 22:37:52 UTC
Adding accounts@

Also, did you try using keymaster.freebsd.org to replace your key?  If you ssh there with your old key, it should allow you to replace it with a new key.
Comment 10 david 2017-11-05 23:08:23 UTC
To elaborate a bit on the theme that Sean suggested (as well as provide an accounts@ response): please see https://wiki.freebsd.org/clusteradm/ssh-keys.
Comment 11 Mikhail T. 2017-11-06 01:50:17 UTC
> did you try using keymaster.freebsd.org to replace your key?

Nope. Never heard of it until now. Trying to use it today, I get (for whichever key I attempt to use):

I dont recognize your old ssh keys or you have special key constraints, contact accounts@

> see https://wiki.freebsd.org/clusteradm/ssh-keys.

Oh, so it was a while in the making and I didn't know, because I'm not subscribed to developers@

So, what should the new key(s) be like? Algorithm(s) and the minimum size?
Comment 12 Mikhail T. 2017-11-06 02:00:01 UTC
Can we do the "desperation" method, please? Here is the new key (RSA 2053 bits):

http://aldan.algebra.com/~mi/tmp/id_rsa2053.pub

The MD5 of it is 1da7b6e1eee87a0713a76f51a6fd44ed, and the SHA1: 0d5fad12d5728c75dc2913f567a29aca92c9876b

As things stand, I can't even e-mail @FreeBSD.org -- our new ISP has no reverse DNS functioning :-(

Thanks!
Comment 13 david 2017-11-06 02:06:11 UTC
I'm getting clarification re: keymaster's behavior when it encounters (e.g.) a key that starts "command="bin/rdistd -S",no-port-forwarding,no-X11-forwarding,no-pty ssh-rsa ..." (and several other keys that don't have such restrictions).

Folks do not actively "subscribe" to developers@; one is subscribed if one has an active commit bit in the src, ports, or doc repositories.

We can do "desperation" -- but to do that, you will need to "identify a committer who is willing to send the keys, and who is willing to vouch that the keys do actually correspond to you."
Comment 14 david 2017-11-06 02:08:54 UTC
As far as email is concerned: you should be able to send email to postmaster@; please do so, explaining the issue with the ISP (or just point back to this report).  postmaster@ has ways to exempt you from that check.
Comment 15 Mikhail T. 2017-11-06 03:19:09 UTC
> I'm getting clarification re: keymaster's behavior when it encounters

Err, sorry, I don't think, any of my keys had such attributes.

> Folks do not actively "subscribe" to developers@

That's true. But it is possible to active UNsubscribe. And I did that about 10 years ago...

> you will need to "identify a committer who is willing

I was hoping, someone reading this ticket will volunteer as such...

> postmaster@ has ways to exempt you from that check

Good idea... First I was hoping, the ISP will get its act together. But it's been weeks already... Thanks!
Comment 16 Peter Wemm freebsd_committer freebsd_triage 2017-11-06 04:06:45 UTC
FWIW, developers@ is our *only* way to reach people with **need-to-know** information.  If you discard it and you get burned as a result, then that's on you.

There were multiple HEADS-UP emails on the subject. However, as a quick catch-up, here's a summary page with information:

https://wiki.freebsd.org/clusteradm/ssh-keys

David - you can clean up the from= and commands= constraints to allow him to use the self-serve system to update his keys.
Comment 17 david 2017-11-06 04:14:30 UTC
Mikhail: I have followed Peter's suggestion, and removed the 'command="bin/rdistd -S",no-port-forwarding,no-X11-forwarding,no-pty' restriction from the one key that had it.

Please re-try using keymaster after 04:25 UTC Monday, 06 November -- and report the result.
Comment 18 Peter Wemm freebsd_committer freebsd_triage 2017-11-06 04:21:33 UTC
FWIW; finger user@freebsd.org was shut off at core@'s request.
Comment 19 Mikhail T. 2017-11-06 04:28:33 UTC
> developers@ is our *only* way to reach people

Not true -- you could've sent a direct e-mail. I don't blame you for not realizing, somebody may have chosen to unsubscribe from developers@, but the regular e-mail is still a valid option: "Hey, none of the authorized keys we have on file for you are valid -- replace them by XX/YY/2017, details at: http://..."

> Please re-try using keymaster after 04:25 UTC Monday, 06 November

Sorry, still noworky:

% env LANG=C date -u ; ssh keymaster.freebsd.org add < id_rsa2053.pub
Mon Nov  6 04:26:52 UTC 2017
WARNING: /home/keymaster/keyadd.sh: exited with status 1
I dont recognize your old ssh keys or you have special key constraints, contact accounts@

> finger user@freebsd.org was shut off at core@'s request

Sad... Was, probably, the last site on the Internet supporting finger.
Comment 20 Peter Wemm freebsd_committer freebsd_triage 2017-11-06 04:43:57 UTC
Bleah - There was a mis-edit in there.  I fixed it and pushed it manually. Please try again.
Comment 21 Mikhail T. 2017-11-06 04:48:07 UTC
Ok, the keymaster worked this time installing the new key. I am not able to login using it yet, will try again later. Thank you!

Could you, please, do a bulk removal of ALL the old keys, that do not satisfy the new requirements? Thanks!
Comment 22 Mikhail T. 2017-11-06 05:29:52 UTC
Logged-in, thank you.
Comment 23 david 2017-11-06 15:20:03 UTC
(In reply to Mikhail T. from comment #21)
I'm pleased and relieved that you got keymaster to work for you.

However, this (Bugzilla) is not a communications channel that is considered "verifiable" for accounts@FreeBSD.org.  It is, therefore, not useful to try to use it for transmission of keys or for requesting that keys be added or removed.

The preferred way such requests are handled is via email that is signed, using the PGP(-compatible) key that is committed to the Handbook (doc/share/pgpkeys/${login}.key), sent to accounts@FreeBSD.org.

As a fallback, the request may be placed in a file on freefall, with an non-verified message (e.g., email -- or even Bugzilla) directing accounts@ attention to the file in question.  (Of course, the file needs to be set up so that others cannot trivially modify it.)