Ask HN: What are the best practises for using SSH keys?
Following the recent post about upgrading SSH keys to ED25519, I am wondering what the best practises (regarding security) for using SSH keys are.
For example:
- Is it better to use a different passphrase on each key, or does using the same one not matter much?
- How much less secure is it to not use a passphrase on a key?
- Should you use a different key per user account, per server, or per use-case (i.e. personal or work)?
- Does increasing the amount of bits in a key really have an effect on the security of the key, or does it not make much difference in a real-world use?
- How/Where should private keys be stored on a device using them?
- What are some of the pros and cons from a security standpoint, and how may doing different things affect the usability of a key?
113 comments
[ 3.2 ms ] story [ 183 ms ] threadYep in an ideal world, though I suspect in practice it doesn't matter much.
- How much less secure is it to not use a passphrase on a key?
You are relying completely on the security of your disk, against either physical or cyber. Use a passphrase, use an agent to manage it.
- Should you use a different key per user account, per server, or per use-case (i.e. personal or work)?
Per client device. This is the device that can be compromised and cause invalidation to be required, so this is the one which should be seperate. For convenience you can maintain all your devices public keys concatenated together and hand them out like that - comment each with hostname and date created for ease of identification.
- Does increasing the amount of bits in a key really have an effect on the security of the key, or does it not make much difference in a real-world use?
Up to a point. RSA in 8 bits is trivial. Go for a highish key length, different key types have different recommended lengths. Note some machines dont support higher lengths.
- How/Where should private keys be stored on a device using them?
In $home/.ssh
Permissions should be set for only you to read with no writing. Even better if your home drive is encrypted as it is only vulnerable whilst you are logged in.
- What are some of the pros and cons from a security standpoint, and how may doing different things affect the usability of a key?
If you hop machines a lot key per client can be problematic. In this case a portable secure drive is useful. Of course one leak can be fatal here.
Try not to fall back on passwords, they have nothing like the same security.
Most usability issues are caused by the people running the servers not reacting in a timely fashion to key updates.
That's a reasonable compromise, since you carry the OTP generator around with you.
My favoured solution is to use a yubikey via gpg: with this method you use your gpg subkey as an ssh key. The yubikey 4 supports RSA 4096 bit keys, if you need NFC then the Yubikey Neo supports max RSA 2048 bit keys.
To be clear, GPG encrypted is better than plain text for all sorts of secrets, but using a single account and passing around passwords won't scale. (Or rather the security implications of everybody having root access via a single shared account and passing the password around should frighten you.)
Ideally there's some sort of mechanism so all users can login to their own account on servers (from puppetized ssh keys to LDAP, among others), and no one is actually passing around passwords.
[1]: https://www.techdirt.com/articles/20160515/02094934446/bad-n...
This is harmful advice in my opinion. A key has sufficient entropy to resist brute force attacks and is never directly exposed to a server. The same key between multiple servers is fine.
- Is it better to use a different passphrase on each key, or does using the same one not matter much?
If they are being used on different device then different passphrases makes sense otherwise no.
- How much less secure is it to not use a passphrase on a key?
10? Not sure you can really quantify an answer. I'd recommend a passphrase if you aren't already using disk encryption with that it's probably less of a concern however with agents there's not much issue with not having one.
- Should you use a different key per user account, per server, or per use-case (i.e. personal or work)?
Use a different key per client device but you don't need a different key for logging into different servers unless you care about people correlating those users.
- Does increasing the amount of bits in a key really have an effect on the security of the key, or does it not make much difference in a real-world use?
Use more than 2048bit for RSA/DSA beyond that it doesn't matter.
- How/Where should private keys be stored on a device using them?
~/.ssh on some local filesystem.
It is better to use a different passphrase for each key but it is also less convenient unless you're using a password manager (personally, I'm using KeePass)
> - How much less secure is it to not use a passphrase on a key?
That depends on the security of the computer where the keys are. I remember a Firefox vulnerability where one site exploiting it was looking for ssh keys on the local file system. So I'd say that a passphrase is very important.
> - Should you use a different key per user account, per server, or per use-case (i.e. personal or work)?
Personally I'm using a key per account per host.
> - Does increasing the amount of bits in a key really have an effect on the security of the key, or does it not make much difference in a real-world use?
Yes up until 2048 where the returns of increasing the amount of bits will start diminishing.[1]
If you are going to use a password manager to store them there's no point in using different passphrases.
Also, the keys don't have to reside on the same device the password store is on. If you have one key per client device for example.
So to protect against that, don't use a password manager. Which means that you're more likely to re-use passphrases, because lots of them are hard to remember and keep straight. So breaking one gets access to many others.
My head spins.
Sure, in general, you want to avoid password reuse to isolate password compromises / security breaches (some companies are still storing passwords in plaintext in 2016, ugh). But in this case, we're talking about private keys that should never leave your local machine. If those are compromised (even passphrase-protected), then it's not inconceivable to think that your password store might also be compromised.
Yeah, reliazed that, my post was just my "logical" progression.
Edit: Feel free to Google "fail2ban security" if you disagree.
I've been bitten by this a few times.
It's better to think about specific attack scenarios. If your keys get exfiltrated because of some local exploit (like a browser vulnerability, a malware download or physical access) then the attacker has access to your servers.
http://m.theregister.co.uk/2013/01/25/github_ssh_key_snafu/
OTOH, it's defense in depth, and there are some vulnerabilities which could allow to exfiltrate keys without non-sandboxed-RCE
https://blog.mozilla.org/security/2015/08/06/firefox-exploit...
I reckon that a passphrase protected key would be safe from this attack (modulo offline brute-forcing)
Yup. IMO not using a passphrase is nuts, considering there is practically no convenience loss (ssh-agent, OS X keychain, pageant) and fairly significant benefit. Protects against:
- accidentally exposing the private key (it happens, accidental wildcard in the shell), and
- attacks that aren't after your keys specifically, e.g. your home directory gets stolen.
I have since discarded these products (and others should too), but they do exist.
Why are ED25519 keys better than RSA, how compatible are they with different systems, and are they worth the hassle and risk of generating and using a new one?
OpenSSH already has support built in, as does Mosh, and therefore, I do think it's worth it.
I do sometimes disagree with using keys though, despite everyone pushing it so hard, there are cases where I think passwords are the way to go, but thats a long post for another time.
Also, I really like usin pam auth mods, such as yubikeys or google auth for two factor. That combined with regular hardening, and yes Ill admit it, non standard ports (reduces log alert fatique) does a pretty good job.
That sounds interesting, could you elaborate on that?
Two reasons: 1) they are a lot shorter for the same level of security and 2) any random number can be an Ed25519 key. To generate an RSA you have to generate two large random primes, and the code that does this is complicated an so can more easily be (and in the past has been) compromised to generate weak keys.
What I don't get then is how can a short key be secure, that goes against what I was taught in college. Aren't shorter keys more prone to collisions and bruteforce attacks?
Given the same cipher, more or less, yes. Between ciphers, though, key-lengths are less relevant, and the differences in those ciphers become more so.
EDIT: Think of it in terms of Shannon Entropy: because RSA requires a pair of primes, the keyspace is so much sparser — that is to say, more "predictable" (if, granted, at a mostly theoretical level) — so keys need to be that much larger to be secure.
Contrarily, with ED25519, keys can be smaller, because the keyspace is denser.
(Or am I just talking out my ass here?)
EDIT 2: s/smaller/sparser/, s/bigger/denser/, regarding keyspaces. Thanks, 'lisper!
The other factor (no pun intended) that makes RSA keys large is that there are more efficient algorithms for factoring than there are for solving the elliptic curve discrete log problem, e.g. https://en.wikipedia.org/wiki/General_number_field_sieve If you crunch the numbers on this you will find that a 2000-bit RSA key has a security level of about 100 bits, i.e. it takes about 2^100 operations to factor a 2000-bit RSA key using GNFS.
Exactly. Generating random primes is not terribly difficult in theory, but in practice it is very tricky, which makes it hard to answer the question: how do you know you can trust your keys? Sure, you can verify that your primes are prime, but how do you know how much entropy they have? The only way to figure that out is the audit the code. (And then you have the problem of making sure that the code you're running is the code you audited.)
Generating random numbers is also tricky, but a lot less so than generating random primes: take an entropy source and run it through a whitener, i.e. feed it to sha512. As long as you have a reliable estimate of the lower bound of the quality of your entropy source, you're good. A lot fewer moving parts.
ECC and RSA are both ultimately going to fall to practical quantum computing, and probably over a time frame short enough that it won't matter which cipher you picked.
All that being said, thats a certain level of paranoia which i dont think is productive.. but perhaps for targets of the state or what have you, I'd rather not chance it.
AFAIK the main benefit is: if you want to run a multi-user or headless system, you need the network stack brought up before a password is entered - which home directory encryption provides and FDE can't.
It does not give any extra benefit because tmpfs only resides in RAM, and disk encryption typically only concerns data at rest.
If you come to a point where you want to have /tmp/ encrypted, you would rather encrypt the whole RAM. (Does anyone actually do that?)
To put it in perspective, if you were logged out completely and tried to login remotely, you would be forced to use a password login initially. If it was more like my desktop where I'm logged in and the screen in locked, the home folder is unlocked, so I can remotely ssh in using just my keys.
Upgrade your SSH keys! https://blog.g3rt.nl/upgrade-your-ssh-keys.html?_utm_source=...
http://javacardos.com/store
Or if you have a friend that knows Russian you can use this http://www.smart-card.ru/_catalog/?search=jcop
Passphrase is strongly advised
This is a question of layers. If you don't have a passphrase on your key, what stops someone from gaining access to it? Just your account password? If they steal your device, is there some form of storage encryption involved?
> - Should you use a different key per user account, per server, or per use-case (i.e. personal or work)?
I have different keys for different purposes per client device. This is mostly because sometimes I need to login to places that are ancient enough I need to use a weaker key than I would like to use in other places or vice-versa, there's places I can only login with ed25519 keys.
Though having different keys per purpose isn't necessary it allows me to keep certain identities separate. I have a different one for GitHub for example, mainly because GitHub exposes my public key and therefor allows for clever tricks like tying the key to an established identity should I use that key to authenticate in other places.
I would also recommend configuring SSH so that it doesn't send over any/all keys by default. Take a look at the IdentitiesOnly option in ssh_config.
From that, we get:
If they only allow you to upload one public key per account, then you aren't able to use that account on both your desktop and laptop, if you have separate private keys on them, no?
Edit: Actually I just looked at github and they allow multiple keys. Is it generally the case that any service that offers ssh keys will allow multiple keys?
Regarding key types:
- DSA keys (ssh-dss) suffer from several issues (fewer bits, bad RNGs in Debian, other issues), and modern versions of OpenSSH deprecate it.
- RSA is pretty standard, and generally speaking is fairly secure for key lengths >=2048. RSA-2048 is the default for ssh-keygen, and is compatible with just about everything.
- ECDSA is largely considered compromised because the constants NIST chose for the cryptosystem weren't well documented how they got them, and the assumption is that the NSA chose them to provide a "backdoor" (so it would provide the same security for a general attacker, but significantly easier for them). This was confirmed as being theoretically possible, and there is of course concern that the NSA could potentially leak those constants, instantly breaking the security of this cryptosystem.
- ED25519 is more or less the same as ECDSA, but was put together by DJB. The big advantage here is speed. EC crypto is much faster to sign, slightly slower to verify, and equivalent security can be achieved with fewer key bits.
- Notes for the future: both RSA and ED25519 become insecure against quantum computing (integer factorization and discrete log are both in BQP).
Generally, use RSA if you work with older servers that only support it, or ED25519 if you like shiny things. Otherwise it's a bit of a tossup.
Regarding using separate keys:
- I follow the philosophy that a private key should never leave the host it was generated on. If you aren't sharing keys between machines, you remove the risk that you'll accidentally share it publicly.
- Beyond that, I'd recommend at a minimum having separate work/personal keys. Keeping separate keys for each user/host you want to log into is a tad excessive, but can be useful for key revocation/rotation.
Regarding passphrases on keys:
- Yes. FDE is sometimes trivial to bypass, and you want to be protected in case someone sets your ~/.ssh folder to be synced to dropbox/samba/etc. You can use an agent to keep the decrypted keys in memory, but I'd avoid using agent forwarding.
Regarding bastion hosts:
- You didn't ask about this, but it is essential for a "best practice" setup.
- Bastion hosts are small VPS hosts that basically run sshd and have a static IP. You disallow any ssh traffic except from your bastion hosts to your servers.
- You'll want to have at least 2 bastion hosts with different hosting services, in case one isn't available.
- Run sshd on your bastion host on a port other than 22. Not for security, but for reducing log volume.
- Run fail2ban on your bastion host, even if you've disabled password authn. Again, not for security, but for reducing log volume.
- Set up fail2ban to alert when a new IP successfully logs in.
Other stuff:
- SSH can use certificates for authentication, and this can make the key distribution problem much easier to solve. I have a script that makes this easier.
- Push for everyone in your organization to use SSH keys, and only SSH keys.
- Defense in depth. All it takes is skipping one step and you expose yourself. Assume that something that was exposed has been compromised. An attacker only needs to succeed once.
tl;dr - the defaults are fine and password protect your keys.
http://lackof.org/taggart/hacking/ssh/
use the defaults, lady.
With large scale hacks in mind and apropos of the Yahoo hack, an attacker has miniscule chances of getting into your account, but 500 million accounts / a 6-digit auth code means with a full compromise (ie, username and plaintext passwords, which the Yahoo compromise was not), the attacker would still get into 5 accounts.
Given how much full email access is "keys to the kingdom" given password resets, thats the (tiniest) bit worrying.
It supports OCSP responders as well as CRLs.
It can allow or deny access based on X.509v3 extensions.
Using this and a hardware token with an X.509 certificate you can completely eliminate passwords for most cases and have no ability to leak your keying material (barring HSM vulnerability).
Android and iOS devices have historically had little support for smartcards so that's a downside of you use those devices. There's a smartcard framework in Android but I've not used it since 4.x and it was not useful.
Support on ChromeOS, Linux, Windows, Mac OS X, Solaris, and *BSD has been a lot more reliable.
Also, this is a fairly common usage scenario thanks to HSPD-12 and the tens of millions of affected PKI users.
I have setup a VPS, disabled passwords, and setup a key with a passphrase to gain access. At this point my greatest worry is losing this private key, as that means I can't access the server.
What is a reasonable way to backup my private key?
Encode it as something similar to a QR-code, print it, and store it in a hole in the wall? Copy it to an USB-stick and hide that somewhere safe?
Alternatively, I have access to more than one computer, so I could also authorize a couple of other keys to access the server. So I would transfer the public key to the authorized machine, and add them to the authorizedkeys from there?
How to deal with the possibility of death? Do I trust someone with my keys and passphrases?
Edit:
To elaborate on this a bit more -- perhaps I'm not an ideal role model for 'best practice' ssh key behavior -- but I end up dealing with a whole load of keys and boxes in $dayjob:
punkz:~$ wc -l ~/.ssh/known_hosts
1473 /home/punkz/.ssh/known_hosts
punkz:~$ ls -l ~/.ssh/id_* | wc -l
31
I deal with these by creating a new keypair for each client I'm working with (sometimes one for each of their environments) and I incorporate their name, or the location of the office I first attended for them, into the passphrase for the key somehow. While this is similar enough to the others that I won't forget the whole thing and so if I have to pick up some old client where I've forgotten the phrase there's usually only about 4 combos with the client's name or the address with various CaSeOpTiOnS or so on that I can usually get there..
I suspect if someone, or ... say .. some state, was targeting me personally to get access to a specific key that if they've gotten to the point where they have copies of them anyway; I wouldn't be able to stop them with a clever passphrase which is more unique than what I currently do.. Thinking along those lines then, I'm aiming to protect against lazy/stupid attackers wanting 'low hanging fruit' as it were, vs the big boys who probably have other ways of pulling these secrets from me....
I maintain a completely different passphrase schema for my personal keys -- either way, that's how I do it; seems good enough for me.. Any thoughts?
Or say, someone pwning a dev jenkins box doesn't get an agent connection which lets them into prod...
Look at ssh's -W or -J options (depending on how old your ssh client is), or see e.g. [0].
[0] https://en.wikibooks.org/wiki/OpenSSH/Cookbook/Proxies_and_J...
As for access if you lose a key, unless you somehow run a VPS that also uses full-disk encryption (and requires a passphrase on boot), then your hosting provider has access to your VPS.
For instance, with DO or Linode, you can log in to the admin console and reset the root password of a server. Once done, you can use the terminal built into the admin panel to access to the server even if SSH is disabled.
Presumably, someone could work with the provider, prove you are dead, and that they should gain access to your servers. At that point it would come down to the policy of the hosting provider.
The disk encryption doesn't buy you much in such an environment, though... Even with whole disk encryption if you're running virtualized; then your provider can just read your VM's memory anyway...
Unless your VPS is going to be forever shutdown, disk encryption under a hypervisor you don't control seems reasonably pointless -- I only use it for physical security to stop people being able to use disks they yank from physical boxes -- even then if they were motivated enough they could probably grab the memory from the server(s) and get the keys anyhow...
AFAIK, there is no possible protection from this besides running your own gear. There were recently published attacks where someone managed to pull keys from other machines by abusing some either intel or vmware's memory dedupe tech bugs iirc (don't have the link handy) -- so; it's not even limited to what your provider may be doing...
I really must have missed the story here -- people pay for a third party service to manage their private keys?
... Err. That's so illogical it's making my brain hurt to even work out why I would need to explain how illogical it is...
I can't even come up with a good analogy for how wrong that whole idea is, fortunately I don't have to though since no one that has mastered 'ssh-keygen' and 'cp' would actually do so.....
2FA, okay; the service provider only has half the story. ssh-keys-as-a-service? WAT.
Edit:
Okay, I looked a bit more at userify and it seems to be an agent that runs on your systems which generates authorized_keys files....
From their docs:
"There are two parts of the Userify installation: creating the agent (readable only by root) in /opt/userify, and setting the agent to start at bootup. Setting the agent to start at bootup is a bit trickier, but the installation process attempts to detect the server type and set it up properly."
Am I reading this wrong? You hook up an agent, which runs as root, which farms out authorized_keys entries across the estate? One possible mech for deploying the agent is conf management?
What the actual fuck? Why would anyone do this? Do I even need to explain why this is blowing my mind so much?
I'm not saying it's a good solution though. I agree that it's giving someone full access to your whole environment and it's a bad idea.
Okay, there are a few things here.. I'm not trying to rant at the people behind this directly, there are a number of similar agent-based tools which have similar issues, so please don't take it personally...
1) If you have the ability to roll out an agent such as this, with your conf management, then you could just manage your users that way instead....
2) Even assuming that this SAAS is un-hackable (lolz), you're now in the situation where all access to every machine in your environment can be changed with a single password on some web ui you don't control, which can be accessed from anywhere.
3) This should go without saying -- I can't put too much faith in how well it's implemented -- pwn the saas, then I immediately own every system connected to it..
4) The enterprise docs talk about sitting it next to your LDAP system... If I had an ops discipline and environment which has gotten to the point where I'm dealin with LDAP, why wouldn't I just configure PAM to read my ssh keys from that, instead of this?
I understand this is one of many such products which are coming out of the lack of desire/time/staff to handle opsy-style tasks when features could be getting written (I'm planning a talk on this...)
But.. does anyone really think that the cost of spending a day wrangling ansible to manage access to your servers is worth more than giving some random third party the ability to add users to your production databases?
I've dealt with many, many different auth mechanisms up to this point. LDAP+Pam_MkHomedir, NFS, Config Management, Shared Keys, Shared Passwords, etc etc. Each have problems, but I don't think the issues with any of those are solved by this, are they?
This must be aimed solely at small startups with no ops experience, right? No one would actually put this near real data, would they?
I hope so. But if I'm wrong, I'd like to hear from the actual users what do they get out of it.
Not to me, I felt exactly the same way.
I simply cannot fucking believe that somebody would "outsource" something as sensitive as this. Then again, I don't run my services on other people's computers either. My servers are either in our cages in datacenters or in buildings that we own, they're locked down as much as I can get them (DISA STIGs, etc.), we encrypt the hell outta data, we have strict security policies, etc.
I really can't imagine doing something this crazy but clearly I'm in the minority.
Also, you're probably running servers at a third party hypervisor already.. ;) they could be scanning your RAM and reading your disk and you'd never know.
I certainly wasn't trying to shit on you (or your company/employer) personally... but to me the idea of turning over ultimate control of all my servers, data, etc., to some unknown third-party and simply trusting them is absolutely fucking absurd.
I pretty much agree with everything cyberpunk said in this thread. I've been around long enough that it shouldn't but some of the things people do with regard to security sometimes surprises me and just leaves me standing there with my mouth hanging open, saying "WTF?".
So how do you feel about TLS and the 100's of trusted certificates embedded in your browser/mobile device?
... eh?
EDIT: Case in point.. did you know the name WoSign before they became notorious? https://docs.google.com/document/d/1C6BlmbeQfn4a9zydVi2UvjBG...
Using a passphrase is highly recommended except for server-to-server accounts, which should be locked down (and specify the specific command that server can execute in the authorized_keys file - Userify[1] supports this).
You should definitely use a different passphrase for keys stored on separate computers, and it's not a bad idea to use a different passphrase for separate keys stored on the same computer, especially if they have different servers they can access. However, practically speaking, if your computer was compromised (ie keylogger etc) then it's game over anyway.
> Does increasing the amount of bits in a key really have an effect on the security of the key, or does it not make much difference in a real-world use?
Yes, it does make a difference, depending on what you mean by "real-world". Anyone less than a state-level actor will probably be unable to cost-effectively attack even a 1024 bit key, but that won't be true for long. We suggest 2048 bit keys if you are using RSA, with 4096 if you prefer extra security and don't mind slight latency during a connection, or ED25519 for keys on systems that support it. Generally the defaults are pretty good. We have a HOWTO for different OS's here: https://userify.com/docs/generating-ssh-keys-on-ec2/
> How much less secure is it to not use a passphrase on a key?
From the server's perspective, it's EXACTLY the same, but from the client (your laptop's) side, it's completely different. While it's possible that your laptop could still contain your decrypted key in its key manager's RAM or suspended state (ie unencrypted swap file etc), the use of a passphrase even on (actually, ESPECIALLY on) a non-full-disk encrypted system will raise the level of effort to access your key to near-impossibility levels, especially from non-state actors, whereas a key that has NO passphrase is a piece of cake. Use a passphrase EVEN WITH full disk encryption (for example, the evil maid attack)
> Should you use a different key per user account, per server, or per use-case (i.e. personal or work)?
If you're using a different key and storing them on different computers, you should probably use a different passphrase on each key. The passphrase (or even if one exists) is not visible to remote servers (or Userify[1] - we provide a free-text field that becomes your authorized_keys on remote servers.)
You don't need to use a different key per user account, although you can. You also should not use a different key per server.. that will turn into a management nightmare. It's perfectly ok to use one key everywhere, but you should probably use a different key on your laptop and desktop, or if the keys have different levels of access (Userify[1] can automate that for you too).
> How/Where should private keys be stored on a device using them?
Ideally on a device using full-disk encryption, including swap and laptop suspend space, to prevent access to a decrypted key in RAM (you are using a passphrase, right?). However, FDE does not protect you from other compromises on your system (i.e., another user that gains escalation to root and installs a key logger), and does not protect against a compromise of your BIOS (i.e., Intel UEFI) or boot process (evil maid attack again).
> What are some of the pros and cons from a security standpoint, and how may doing different things affect the usability of a key?
Keys are safer than certificates because there are less moving parts and no outside requirements for your internal CA or dependency on a CA that might go down. Keys can be a management nightmare at scale, but there is software to manage them (ie Userify[1], ManageEngine[2], BeyondTrust[3], ssh universal key manager[4], keybox[5] (free&#x...
1) Disable passwords and only allow keys even for root with PermitRootLogin without-password
2) public-key authentication has somewhat unexpected side effect of preventing MITM per this security consulting firm http://www.gremwell.com/ssh-mitm-public-key-authentication
If you expect to be moving your SSH keys across machines (e.g. to use your same personal key on both your laptop and your desktop), then they should absolutely be passphrase-protected, even if they're only transferred via encrypted media.
There's not really a right or wrong answer to this besides "don't reuse the same key everywhere". I personally maintain one key (really two: one RSA, one ED25519) for all of my personal devices, and maintain a strict policy of full-disk encryption on such devices. I've occasionally maintained separate work keys so that I'm not ever in a position where I need to make my personal keys available to an employer.Meanwhile, for situations where a server needs to connect to another machine via SSH, each such server gets its own key. That way, if a server is compromised or decommissioned, I can revoke access by key.
It makes a significant difference. More bits → exponentially more attempts required to brute-force it. Depends on the device.A reasonable balance between security and practicality is for any portable media (including portable devices, like laptops/tablets/phones) to be encrypted (in addition to the key itself being passphrase-protected). Better security would be to extend this to non-portable media and machines as well (but this is painful to enforce on servers unless you have physical access).
The directory in which keys are stored should only be accessible to the OS user actually using those keys (so, for example, `~/.ssh` should have permissions `drwx------` when viewing with `ls -la`).
Basically, server SSH keys should be treated like you'd treat your SSL/TLS keys.
...of? Pretty much everything involves security v. convenience tradeoffs. Generally, the more secure, the less convenient, and vice versa. While absolute security is ideal, a lack of convenience makes it more difficult to effectively enforce (e.g. as part of a company-wide security policy) unless you're willing to put in the work to build up an effective workflow around it.