Poll HN: Do you use SSH certificates (not mere public-key authentication)?

Infos about how to use them would be great.

I've used public-key basically forever.

I have wanted to go that step further and implement certificates for a good few years now, I absolutely see the value, no question about it (i.e. no more out of date authorized_keys files, no need to even install authorized_keys files in the first place, the list goes on).

I wish YubiKeys were more flexible in allowing SSH certificates to be hosted on them (currently can only host X509 type). Because if I could use a Yubi for SSH certificate signing I would have already been using certificates ! (Yes I know, theoretically you can do it with GPG keys, but frankly that's a PITA with many issues).

My current plan is to take inspiration from this : https://smallstep.com/blog/diy-single-sign-on-for-ssh/

  ssh-keygen -D opensc-pkcs11.so -e > cert-on-piv.ssh
  ( echo -n "cert-authority "; cat cert-on-piv.ssh ) >> ~/.ssh/authorized_keys
  name="temp-$(date +%Y%m%d-%H%M)"
  ssh-keygen -q -N '' -C "${name}" -t rsa-sha2-256 -f ${name}
  ssh-keygen -t rsa-sha2-256 -s cert-on-piv.ssh -D opensc-pkcs11.so -V -1d:+1d -I ${name} ${name}.pub
  ssh-add -t 1d ${name}
  rm -f ${name} ${name}.pub ${name}-cert.pub

What's the difference between a certificate and a public/private keys?

I've set them up for both identification and authorization with most of my virtual machines. I've set up two CAs (one for server identification, one for client identification) and provision virtual machines to trust the client authorization certificate by default. I still need to work something out for setting up server identification automatically but I haven't had the time (or interest) to find out how to set proper random hostnames through cloudinit scripts.

I think the certificate system is especially great for certificate login from multiple devices. At some point I was just constantly copying/updating authorized_keys files and with SSH certificates I can just provision my clients once and the problem is solved, assuming I use the same username everywhere.

On the server side it's kind of annoying to set up right because there's no easy process like ACME for SSH certificates. Depending on how much effort I want to put into (temporary) virtual machines I'll either TOFU the host certificate or go through the effort to properly sign it if I have easy access to the CA key from the device I'm working on.

We use an AWS KMS asymmetric key for the CA keys, they're cheap and avoids exposing the private key material in an any way.

For signing SSH certificates, we run a small service (prototype code dump at https://github.com/pardot/sshsigner) that uses this key to sign short lived certificates. Auth to the service is via OIDC issued ID tokens.

On the client side we have a custom SSH agent that uses an ephemeral in-memory private key. The agent manages the OIDC web flow and calling out to the service for signing on demand. This lets us keep the cert duration small and scoped, and allows us to force re-auth for sudo etc. via the web flow.

We also do a similar thing for host keys, IAM auth the instances and sign certificates.

Altogether works well, provides a nice user experience, and keeps long-lived/leakable creds out of out environment.

I use Kerberos (KDC hands out TGT only after MFA)

We looked into them once at work for our "conventional" SSH infrastructure, but immediately rammed into the wall that a certificate can sign a key, but you can't have a full PKI infrastructure like SSL where certs can sign certs that can sign keys. Without the ability to delegate, the logistics of needing to create an absolutely centralized authority to do the useful things we wanted to do within the organization raised the cost of "first value" too high for us to fuss over. So for large-ish organizations (~1000 affected employees) we had a hard time justifying any effort put into it versus the established infrastructures for key management that already exist.

On the other side, as a personal user of SSH with basically one person to worry about, the effort of setting up a certificate seems like just a waste versus the existing key-based infrastructure; I don't understand at all what attack it would prevent or what convenience it would provide for the cost of learning it.

A general principle is you get good at what you do a lot, and you aren't good at what you do rarely. Key-based management has its issues, but people have been doing it for a while and they're good at it. Not necessarily "great" and not necessarily "unable to be improved", but they're good at it. People understand it and know how to use it. Nobody uses certificates, so nobody understands how they work or what to do with them, and they have the weakness I cited above. So, even though in a perfect world certificates may be superior, in the world we live in now, the competition between the existing polished infrastructure for pure key-based management is pretty good, even if you think it's a polished turd, and the infrastructure for SSH certs is pretty bad, even if you think it's an uncut diamond in the rough. It's difficult to look at that situation and honestly choose to proceed with the certificates.

As others point out, the cloud services are proceeding in other directions as well which will probably continue to make using SSH certificates directly a very difficult choice to practically justify.

I recently got SmartCard HSM (also available as Nitrokey HSM 1 and 2 I think), but didn't get around to setting it up yet. For personal use it does seem like an overkill, when FIDO2 security keys are much more convenient.

If you use Google Cloud’s “OS Login” feature, you’re using (semi-ephemeral) SSH certificates.

I store my authorized_keys in DNS TXT records, that are DNSSEC signed, with a validating resolver on the box. I then just use "/usr/bin/hesinfo %u ssh" as my AuthorizedKeysCommand in OpenSSH.

I wrote a little tool that allowed you to "#include" other DNS records etc, but "hesinfo" is generally easily installable/available so it's just easier.

    fn authorized(peer: ClientConnection) -> bool {
      return true;
    }

I use them for personal machines, and have deployed them in the past in work environments. A few years ago I wrote a CA which can exchange oauth tokens for signed keys: https://github.com/nsheridan/cashier

Auth is handled in a browser e.g. by Google, and the CA will sign a key and return a cert with a valid token.

I use private keys but store them on a Yubikey. In OpenPGP format, not using Fido2 at present. I use OpenPGP for other stuff too like my password manager so I'm not ready to move to fido.

But certificates - no. I don't see the need for this in my home environment. In my opinion it only makes sense if you have a PKI, in which case you already have the whole CA scenario covered.

I use certificates to deal with the n clients -> m servers i have privately. On some machines i have AuthorizedPrincipalsFile include my username, so i can directly login as root there.

I found these SSH features while studying the ssh-keygen(1) man page. I don't use extra tools aside what SSH ships.

I did for a while, then didn't because neither my chromebook nor my android devices supported them. Then they did and now I do again.

Hypothetically, SSH over SSL/TLS or an iodine tunnel along with a few IDS tripwires.

Every approach has strengths and weaknesses, the password-less method is convenient... but is the number one target for worms/bots. =)

I use public/private with the private key handled by Yubikey PIV functionality.

Doing this without something like Vault is very difficult. Vault is designed to provide similar guarantees to an HSM (it's a service designed to make keys usable but not exfiltratable), and that's essential for any long-lived certificates.

Bearing in mind that, on principle, I never expose ssh to untrusted hosts/networks, for many years my solution for ssh-SSO was GSSAPI. Typically this would be a two part auth, where gssapi-kex had to succeed, _and_ the connecting user had to supply the kerberos password on the server side via PAM/keyboard-interactive to reduce the risk of "sysadmin walked away and left screen unlocked."

I think this approach has fallen out of favour these days compared to PKI, but I haven't been an enterprise sysadmin in a few years.

Edit: for those curious, ephemeral access in this case is easily enough managed via e.g. the usual AD-on-*nix methods. It doesn't scale beyond a few dozen systems, but the scenario described above was back before scale-out became the fashion.

I don't see the benefit of using certificates for personal use. In my mind, certificates only really shine when administering a fleet of servers in a large company, as it eliminates the need to update the authorized_keys file on every server someone is hired or leaves.

I could be missing some less obvious use cases, but if you're just authenticating to your personal machines, it doesn't seem worth it. It's not any more secure than SSH keys, and you have to also manage and secure your CA key.

For those curious about how SSH CAs and key signing works, I wrote a blog post about that awhile ago: https://dmuth.medium.com/ssh-at-scale-cas-and-principals-b27...

I also created a "playground" which can be used to stand up such an environment running in Docker containers to better understand the process: https://github.com/dmuth/ssh-principal-and-ca-playground

Hope folks here find them helpful.

My hope is to make this problem mostly redundant among my machines through use of tailscale. Tailscale SSH means that I don't have to worry about SSH keys at all - it just uses the same auth as the rest of tailscale.

Only downside is that at the moment it doesn't play with SELinux which is a bit unhelpful for me given I mostly use Fedora.

I use X.509 certificates for SSH (via PKIXSSH [0])... but I'm not sure where that goes here

[0] https://roumenpetrov.info/secsh/

No, because I needed to interoperate with Windows and at the time it didn't handle SSH CA certs. Maybe that's fixed now? Anyways, we use Hashicorp Vault's SSH OTP system. Works fine.