This is a good start, netstat -an to see what ports are open, to shut down things that open ports. Turn of xinetd if it is on, etc. There was a much more complete best practices document that came through here earlier.
Many ISPs (and hosters) do support IPv6, and it’s worthwhile to learn a bit about it. If you are running an internet-facing server, I’d say it’s part of being a good citizen to support the latest standards.
To address the parent question, the firewall rules actually need to be a bit different than for IPv4. As an example, IPv6 makes supporting ICMP (for ping and the like) mandatory.
There is a quite straightforward patch[0], and since fail2ban is python, you can apply it on the installed package. Not the best for maintenance, so use your best judgement (and a few tricks to keep yourself safe and up to date), but it's certainly better than being exposed.
The first time I locked down a server I found iptables to be a little intimidating. If you're on Ubuntu then ufw is extremely easy to use - it's just a front end for iptables.
Very useful for any Linode user.. Compare to AWS, I find it hard to manage security on Linode boxes.. Need concept of security groups like AWS configurable and manageable from UI..
I made the mistake of accidentally setting the firewall too strict on a remote server, killing my ability to SSH. A neat little trick I found was to setup a scheduled task to kill the firewall in 5 minutes, and then restart it. If it's too restrictive and locks you out, wait 5 minutes. If you did it right, then kill the scheduled task.
You can just use iptables-apply. It achieves the same thing and is built-in. If you don't confirm the changes after they are applied, because you locked yourself out, they will be rolled back.
There are many more steps used in more serious situations... chiefly, these come to mind: Validating installation media. Validating/upgrading firmware. Installing a minimalist kernel. Securing BIOS and IPMI. Taking an inventory of part numbers and identities to detect tampering at later stages. Locking down network access on the switch to the appropriate MAC address. Determining administrative access methodology and distributing appropriate keys or credentials. Testing.
Probably most servers today are automatically set up, and live in large scale server farms. They probably boot via PXE, using custom auto-provisioning code. Other than physically taking delivery of the unit and inspecting it and logging its presence in inventory, such machines are literally just plugged in and boot up. Probably most of them are preconfigured to PXE boot by the vendor. A strong setup will also override BIOS settings automatically upon boot, ensuring further boots have 100% pretested/tuned configurations.
The big problem I have is securing my private keys. I use multiple devices, and haven't found a secure and convenient way to share the keys across devices securely. I'd love ideas..
I fall on the side of an SSH key identifies a person, not a device. That's what host keys are for.
What's always been missing though is a decent way to keep track of all the places which might be set to authorize a particular key so you can revoke it if you think it's been compromised (which should mean "is under brute force attack to decrypt it's AES encryption").
I suppose with a DynDNS server and a smartphone app, you could setup a web-page which would allow you to request a URL to download your SSH key securely to new devices but this is kind of the problem: there's just nothing out there which lets you keep a proper eye on how your web-of-trust works for this type of thing.
You shouldn't try to share them, imo, and generate a key for each device. Yes, that may be painful to have to copy 5 public keys on each new server, but the day where, say, your laptop get stolen, you don't have to change the private key on each device : simply remove the laptop one.
If you want some kind of centralization, you could only add your keys on a single server, then add this server key on others. You log on first server, then use it to log on others. But that would be dangerous : if the master server crashes, you can't access other servers anymore. Redundancy is good, in that case.
Better way to reduce the need for copying keys around would be setting up a private CA, and signing the individual keys with it. Of course you need to keep the CA key secure, I'd strongly suggest removable media or preferably a real HSM.
I agree that key-for-each-device is probably best practice.
It just doesn't work for me though. I frequently need to log onto servers from new client computers, and the workflow of connect from trusted device, generate new key for new device then use new device just isn't practical.
I've spent a lot of time thinking about this.
I think the biggest threat to security on my servers is brute forcing of ssh passwords. The best solution for that is key based authentication. (I deploy fail2ban as well though)
There is a non-zero threat of my devices being lost/stolen/accessed with my keys on them.
So key based authentication is the way to go, but I just can't get around the practical aspects.
In the end I often rely on long, random passwords kept in a KeePass database I sync (along with Fail2Ban). I've considered keeping my private keys in that too, and to be honest I think that isn't a terrible solution. It is certainly less than ideal, but probably mildly more secure than my current method (ie: really a private key shared across devices is acting as a very long password)
"I frequently need to log onto servers from new client computers, and the workflow of connect from trusted device, generate new key for new device then use new device just isn't practical."
Agree. But wouldn't it make sense then to generate unique keys for all of your devices (still) and then have a key which is your "travel" key and change that periodically?
I keep my travel key in a pen drive, and in practice I change it shortly after every usage. But that's because I don't use it a lot, you'll probably want to change it every few weeks.
Also, if you don't trust the machines you are using to connect, no time frame is small enough. If you really need to connect from them, use a VM (best) or a guest account (not as good) with access to just the stuff you need.
Your private key should be encrypted with a strong password.
If you follow that best practice, you can stop worrying about your laptop being stolen, and start worrying about five dollar wrenches. (http://xkcd.com/538/)
On Windows I use KeePass 2 [1] with the KeeAgent [2] plugin. The keys will be protected by your master password, and it will act like pageant so most tools will automatically recognize and use the keys. You can configure it to automatically let all applications use the keys, or first prompt you before providing the key to the user.
You can keep the database file on an USB stick or even something like dropbox or owncloud [3].
I do the same, and have found it works beautifully. You need my master password, SSH Key, and my SSH Key's password before you can log into my server. And if you somehow manage to get all of that, you need my account password in order to modify any non-user files.
I've never understood the compulsion to restrict outbound traffic on an internet facing server that you do not intend to be used by other (untrusted) people.
If someone is good enough to own you with everything else locked down, they can change any firewall rules completely if they need to, or just tunnel out over an allowed port.
Creating a non-root user then giving them carte blanche sudo rights is similarly odd to me. I'd rather just use root and /etc/nologin (assuming no one else needed a login shell to run).
While I agree with your reasoning (and personally wouldn't bother with such a restriction), there is a good reason and it's that while they're likely good enough to get root, they might not be or simply can't for some reason. And if they can't, then that extra measure could be worth its cost in time and effort.
Or maybe there's a vulnerability in your web app that grants people shell access. In that case you'll still want to lock down their limited normal-user privileges.
Yes but even then you're still not protecting much by allowing HTTP... said user would still be able to download attack tools locally and use them.
Agreed it will still stop a script kiddie (do ppl even use that word anymore? Showing my age!) from running random crap on high ports, but thats about it.
I've seen more compromised boxes than one can shake a stick at. There's all sorts of reasons that blocking egress is a great idea. Compromises are usually automated bots, and no, they're not smart enough to bring down iptables. Even if it's a human that's pwned you, it's frequently a stupid human, or a lazy human. It's just good practice to practice security in depth.
The most important word in your comment is 'intend'. Unauthorized use is never intended (other than in honeypots, but even there it is somewhat intended...).
Egress filtering is important, if you think the chances are large that a user account is compromised a filter table can help. But an even more effective egress filter is one that you run on the router/hardware firewall just upstream from your machines. After all an egress filter on the machine can be disabled by someone that has compromised that machine.
Exactly right re: doing it downstream (or in the case of virtual machines, outside of the guest).
I didn't mean 'intend' in the context of unauthorised use, i was trying to differentiate between a 'single user' ( can't think of a better term, but i'm sure you know what i mean) application server and systems where you purposefully give shell access to other users but want to restrict egress.
Systems like that are only a privilege escalation away from being compromised and are much softer targets. Anything with shell access for multiple parties not known intimately to the owner of the box should be monitored with great zeal.
Blocking outgoing traffic for all applications except the ones that need the Internet can make a difference, especially under Windows. For example, some applications like to phone home for no benefit to the user and there is sometime no option to turn it off.
The option to block traffic per-application in Linux is no longer included in iptables (I think it was the switch --cmd-owner )
Remote exploit of a non-privileged account that needs to phone home to collect the local exploit to get root (after figuring out which one will work for the environment)? Seen it many times.
I've never been a fan of shunning/blocking ip-addresses based on number of wrong passwords. It's to easy to exploit as denial-of-service attack and can be used to lock out legitimate users.
Possibly use a non-standard ssh port, make sure you disable ssh v1 and apply a password policy, allow password logins only for white listed users. Now you should be reasonably safe against brute force attacks and still have a system that is accessible.
If/when you disable root logins through ssh, try to have another way to login as root - maybe a console/kvm switch, prefferably with remote access through a secure network.
you’ll want to lock down SSH access entirely,
and make sure that only you can get in
This sounds like Ubuntu is by default open to everybody via ssh. I find this hard to believe.
The whole article sounds like a lot of fud to me. For example, what benefit is there in creating a new user with sudo privileges instead of using root directly?
Potentially that if the key is compromised, the holder still requires a password before gaining root access. But if your key is compromised, you probably have a bigger problem.
> For example, what benefit is there in creating a new user with sudo privileges instead of using root directly?
Users should run with as few privileges as possible. Logging in as root requires two sets of keys.
Sudo logs all commands. The commands usable when using sudo can be limited, whereas root gives access to everything. Using sudo creates heightened awareness of risks.
Root is a high value target. Giving it a strong password is important, but locking it is better.
> For example, what benefit is there in creating a new user with sudo privileges instead of using root directly?
Also, when a group of people all need access to the same servers, you don't want everyone SSHing in as root. It's much easier to manage team security when everyone has individual users and sudo rights that can be logged and audited.
If you allow some self-promotion: you can take a look at the ansible playbook repo I've been putting up (including some documentation and background in the doc directory): https://github.com/pjan/the-ansibles
In all honesty; the steps in the TO's article are only providing basic security. Pointing the ansible playbooks to your server will further enhance it.
You should also consider changing the SSH port, but that's up to you (in which case you also need to change the firewall settings!)
Wow, let me say that if you're even kind of thinking about Ansible, you should star that. There's some very good use of vars going on, and I'm going to use some of these examples to improve my playbooks. Thanks! If you're thinking about using Ansible in any capacity, star/fork this project!
This isn't intended as a be-all-end-all guide to security, like the NSA aims for. Instead, view it as a quickstart guide for those first five minutes on a new server, or as a starting point for beginners that have no idea where to even look.
I would use a privileged port for ssh (different than 22). In case a hacker owns the process he would need sudo to open another connection if the port is <1024.
If it's a physical server, start by reviewing all BIOS options, if it's virtual one you control, start by reviewing the host environment for the guest.
2) Follow from the bottom to the top
Follow by review and secure the boot process. (Grub password, kernel modules loaded, kernel module options, kernel sysctl options.
Follow by review the services started "even before starting nothing".
Review what is accessible in the system to "anybody".
Review what is accessible in the system to each "user". Many times a "user" is going to be listening to the hostile network in a port.
Review what the system is going to do across the time, what is scheduled, which resources are assigned to which users (and their processes), what is going to be registered.
4) Does not limit or scrub any traffic
It's not that we forget about IPv6
It's that such iptables, on any 1 core VPS, is vulnerable to DoS from a single IP (you don't need any distributed attack to put down that). A syn attack to the port 22 could be enough.
In resume:
When giving general security advisories, lets try that people build a secure system if they follow our tips. Not let open things like ping broadcast from our host, or IPv6
I haven't enabled IPv6 on any of my servers yet, as I can say with 99.9% certainty that my visitors do not know what it is, let alone have it enabled. Additionally, this is intended as a quickstart or beginners guide, and obviously leaves quite a bit out. That being said, it is definitely something you want to consider, and I've added a followup to the post mentioning that.
The problem with IPv6 was not that it could be enabled... it's that it is enabled by default. In all major linux distributions.
That being said, I've read your update, and that is the point that people seeking for "recipes" should get. If they are going to touch ssh or iptables, they need to be ready to explore the documented options (which are many), to validate that the changes are working, to dive into networking, etc.
Amazed that "Install grsecurity" isn't one of the key things listed here. Sure, you have to compile your own kernel, but the enhanced memory protections and many other hardening features make it an excellent addition to your security arsenal.
It's not just "you have to compile your own kernel". It's "you have to compile your kernel EVERY TIME there's a kernel update". If you forget one update and you leave an exploit on your system, you're screwed.
Having said that, I'm surprised Debian and Ubuntu don't have better support for grsecurity. If they'd provide up-to-date packages I'd switch over.
I'm not really "Amazed" as that is going a particular bunkerization route that resists a certain class of attackers that you would not be capable of holding off anyway as someone who this article is useful to in the first place. Properly configuring RBAC is probably slightly above much of the target audience.
What's wrong with passwords ? The info is in your head and in a physical booklet secured at home for backup. Its only weakness is is keylogger.
A secret key protected by password doesn't provide you more security. You can only connect from the computers having a copy of the key. If that computer is inaccessible or dead, or the private key is erased you can't log in. It also is exposed to keylogger.
So I stay with password for now. It should be a secure password of course. Maybe use private keys and a password for backup recovery only.
Password authentication is prone to bruteforce. With public key auth, even if someone has your passphrase, he also needs the private key to break into your server.
Indeed, there's always the risk to lose your private key. That's why it's useful to have several : I have one on my desktop, one on my laptop and one on my mobile. If any get compromised, I only have to delete one public key in authorized keys and generate a new one. That's still open to some attacks, but way less dangerous than simply allowing anyone from anywhere to log in.
That's also better than allowing only a few IPs on port 22, because you'll never now where you will be and what IP your laptop will have when an emergency occurs.
Fail2ban prevents brute force connection attempts. One can also chose a password resistent to brute force by using a long string. A psalm verse, or a sentence from some peoetry which is easy to remember. Or pick a password made of the first letter of each word of this long sentence.
I have one on my desktop, one on my laptop and one on my mobile. If any get compromised, I only have to delete one public key in authorized keys and generate a new one.
Personally, I still don't like SSH keys. Yes, they are "more secure" in the sense of preventing others from breaking into your servers. But they are less reliable in a sense that they increase your probability of being completely locked out when shit hits the fan.
Laptop, desktop and mobile phone? If you can't imagine a situation where you will loose access to all three at once, you either live a very different lifestyle than I do or have a poor imagination.
For me, the expected loss (i.e. probability * damage) of loosing access to an important server because of some fire of flood is far greater than the expected loss of someone guessing SSH port, my login, my password and then the root password.
Well, maybe you live in some place especially prone to natural catastrophes :) I live in France, and I certainly have lesser chances to loose all devices at once than to have a single one compromised.
I don't want to be picky but this isn't two factor authentication in its usual sense. Two factor authentication uses two independent means/media to authenticate. Protecting the private key with a password is required to protect it of beeing exposed when stolen. But if it's stolen then brute force password guessing is for free. Brute force attack against a login password can be detected and impaired, not with stolen private key.
You shouldn't just update, you should update regularly or better yet set up unattended upgrades[1]. Especially for your hobby projects or personal server because odds are that you won't always have the time to act on every security advisory. (Subscribe here[2] to at least hear about them.) Also, if something breaks once in a blue moon, it's not that big a deal.
Fail2ban is fairly heavy and only very recently supports IPv6 (which means the version from your repo may not). You can achieve similar results with something like
Security is a tradeoff. In this case, unattended upgrades lead to increased downtime.
(Why? Because you didn't read the caution note that upstream sent along to you, that's why. You just woke up to the alert saying that your system is down.)
If you have a reasonable environment, you should have alpha or qa servers that look just like production, except that they aren't visible to the outside. Always test changes there before upgrading on your production machines.
Another really nifty way to achieve pretty much the same is to use iptables' built-in limit module like so:
-A INPUT -p tcp -m tcp --dport 22 -m state --state RELATED,ESTABLISHED -j ACCEPT
-A INPUT -p tcp -m tcp --dport 22 -m state --state NEW -m limit --limit 3/min --limit-burst 4 -j ACCEPT
-A INPUT -p tcp -m tcp --dport 22 -j DROP
Limit-bursts allows that many packages through before any limit counter starts, if any of the other packages afterwards also do not garner any response, a counter counts up for every further package, effectively blocking everything after the first 7 packages arriving within a minute. (You could also do 3/hour, for example, if you think you need it.)
The limit module is a beautiful thing, since you can also use it to limit log messages generated for events etc., using the same syntax, like so:
Both limit and spindritf's suggestion will mitigate the attack, but they won't notify you about it. In most circumstances, I'd prefer to skip on the thousands of notifications per day, however I sometimes like to know about every detail on a server, and Fail2ban gives me that level of control, without the need to tail a log file.
As a seasoned system administrator, I strongly advise you against unattended or automatic upgrades. In a Linux or FLOSS eco system components are very strongly coupled but loosely developed. Say glibc is base of nearly every server application but it is and server apps are not regres tested for each other. Which will or may lead to incompatibilities, down times or performance issues even on edge cases data loss. So please don't.
While you are absolutely correct, no one is regression testing the updates potentially coming into their project box/gameserver/etc, even if they perform the apt-get upgrade manually.
Might as well set it automatically and stay protected <shrugs>
I prefer to be actively administrating my server, so that I know if something does go awry. If I'm asleep, it may be a few hours before I realize that an update broke something. On top of that, I clone my servers and test upgrades in a development environment as much as possible, before allowing an update to go live. As long as you're on top of the updates, a few days between automatic and manual shouldn't have much effect.
Your concerns are not an issue with a distro like Debian where the only upgrades are for backported security fixes (or, when an announced point release comes around, critical bug fixes). I worked for several years at a shop with dozens of Debian servers, and nightly unattended upgrades very rarely caused issues, and never caused anything truly serious.
Debian is very stable. I recommend running "safe-upgrade" nightly -- and keeping an eye out for when you need a "dist-upgrade" (typically new kernel images -- and other stuff that should be tested (eg: test that the server actually boots -- not to mention do a reboot to use the new kernel).
I'm not sure how good the Ubuntu LTS relases are, but Debian has always been great at both keeping a stable system up, and keeping the upgrade to new version as painless as possible.
In the typical case, you only need dist-upgrade for upgrading to a new Debian release (which is every 2 years or so and is big enough news you'll probably hear about it). When there's a security update for the kernel, a regular upgrade/safe-upgrade[1] is sufficient to get it. Of course you must schedule a reboot yourself.
[1] "upgrade" if using apt-get, "safe-upgrade" if using aptitude
> Your concerns are not an issue with a distro like Debian
Ten years ago, I had a Debian server completely wedge itself on an update. glibc was hosed, practically nothing would run, and I couldn't untangle the mess. Had it been unattended, downtime would have been even longer than the hours it took to rebuild the box from scratch.
This was not the first or last time such an event would occur, merely the most severe.
Updates break things. Anyone who claims otherwise is either incredibly lucky, or incredibly inexperienced.
But if you can't flip a switch to deploy an identical server and/or restore from a backup image, aren't you in deep shit anyway? Your failure plan is to rebuild the box from scratch?
For a one-off budget server in 2002-2003? Yes, yes it is.
For a highly-available mission-critical infrastructure in 2013, the failure plan is also to rebuild from scratch, because "failure" means redundant and backup systems have exploded. This is most likely to occur when you automatically roll out untested changes to your infrastructure.
In either case, you won't be sleeping tonight. Or possibly tomorrow night.
I used to think that the Redhat security guides were good [1], but would also go through the NSA guide to securing Redhat [2]. Seems a little ironic nowadays.
Recently, we had Bryan Kennedy's "first 5 minutes" [3], linked from Drew Crawford's guide to "NSA proofing" email [4] (pretty good guide to securing a mail server).
LIFS (Linux Iptables Firewall Script) is aimed at what UFW does but it also provides NAT, port forwarding and allows you to work with groups of hosts and services.
What is the purpose of creating a new user and not using root, assuming ssh password auth is disabled, and only I have the key. Making a new user that has sudo an attacker geting that user as opposed to root all they have to do is sudo anyway?
Sudo ask for your account's password, so it's a protection in case someone manage to get your private key (or manage to login with OpenSSH using an exploit).
This article is okay, nothing against the author but the topic of securing a host depends on many factors and it's so extensive at so many levels that this post doesn't even scratch the surface.
The consultant route is a lot easier said than done; at least, if you want a good consultant.
Most truly competent sysadmins already have full-time employment by a product company or hosting provider. Most so-called sysadmin consultants are former web developers who taught themselves Linux and security, and don't know what they don't know.
142 comments
[ 3.4 ms ] story [ 239 ms ] threadnetstat -ntul
(Thanks "child", I did indeed initially not have the -u there. You should.)
.. which is a mnemonic trick, "tulpen" is "tulips" in German...
But home made scripts and rulesets, may or may don't, depends on implementations.
Many ISPs (and hosters) do support IPv6, and it’s worthwhile to learn a bit about it. If you are running an internet-facing server, I’d say it’s part of being a good citizen to support the latest standards.
To address the parent question, the firewall rules actually need to be a bit different than for IPv4. As an example, IPv6 makes supporting ICMP (for ping and the like) mandatory.
I wrote a post with a simple ip6tables setup about 4 years ago (in German, though, and there might be better posts): http://blogs.noname-ev.de/commandline-tools/archives/74-ip6t...
[0]: http://www.fail2ban.org/wiki/index.php/Fail2ban:Community_Po...
https://help.ubuntu.com/community/UFW
http://www.rfxn.com/projects/advanced-policy-firewall/
or
apt-get install apf-firewall
There are many more steps used in more serious situations... chiefly, these come to mind: Validating installation media. Validating/upgrading firmware. Installing a minimalist kernel. Securing BIOS and IPMI. Taking an inventory of part numbers and identities to detect tampering at later stages. Locking down network access on the switch to the appropriate MAC address. Determining administrative access methodology and distributing appropriate keys or credentials. Testing.
Probably most servers today are automatically set up, and live in large scale server farms. They probably boot via PXE, using custom auto-provisioning code. Other than physically taking delivery of the unit and inspecting it and logging its presence in inventory, such machines are literally just plugged in and boot up. Probably most of them are preconfigured to PXE boot by the vendor. A strong setup will also override BIOS settings automatically upon boot, ensuring further boots have 100% pretested/tuned configurations.
The big problem I have is securing my private keys. I use multiple devices, and haven't found a secure and convenient way to share the keys across devices securely. I'd love ideas..
But this doesn't solve the problem you have to connect to each host to reject the lost key. (But you have the same problem with 1 key, so no worse.)
What's always been missing though is a decent way to keep track of all the places which might be set to authorize a particular key so you can revoke it if you think it's been compromised (which should mean "is under brute force attack to decrypt it's AES encryption").
I suppose with a DynDNS server and a smartphone app, you could setup a web-page which would allow you to request a URL to download your SSH key securely to new devices but this is kind of the problem: there's just nothing out there which lets you keep a proper eye on how your web-of-trust works for this type of thing.
If you want some kind of centralization, you could only add your keys on a single server, then add this server key on others. You log on first server, then use it to log on others. But that would be dangerous : if the master server crashes, you can't access other servers anymore. Redundancy is good, in that case.
How do I use a private CA to generate(?) keys for ssh authentication?
It just doesn't work for me though. I frequently need to log onto servers from new client computers, and the workflow of connect from trusted device, generate new key for new device then use new device just isn't practical.
I've spent a lot of time thinking about this.
I think the biggest threat to security on my servers is brute forcing of ssh passwords. The best solution for that is key based authentication. (I deploy fail2ban as well though)
There is a non-zero threat of my devices being lost/stolen/accessed with my keys on them.
So key based authentication is the way to go, but I just can't get around the practical aspects.
In the end I often rely on long, random passwords kept in a KeePass database I sync (along with Fail2Ban). I've considered keeping my private keys in that too, and to be honest I think that isn't a terrible solution. It is certainly less than ideal, but probably mildly more secure than my current method (ie: really a private key shared across devices is acting as a very long password)
Agree. But wouldn't it make sense then to generate unique keys for all of your devices (still) and then have a key which is your "travel" key and change that periodically?
I keep my travel key in a pen drive, and in practice I change it shortly after every usage. But that's because I don't use it a lot, you'll probably want to change it every few weeks.
Also, if you don't trust the machines you are using to connect, no time frame is small enough. If you really need to connect from them, use a VM (best) or a guest account (not as good) with access to just the stuff you need.
I just thought of enabling 2 factor authentication instead of plain password auth, which I think is probably reasonable too
If you follow that best practice, you can stop worrying about your laptop being stolen, and start worrying about five dollar wrenches. (http://xkcd.com/538/)
You can keep the database file on an USB stick or even something like dropbox or owncloud [3].
[1] http://keepass.info/index.html
[2] http://keepass.info/plugins.html#keeagent
[3] http://owncloud.org/
Discussed here: https://news.ycombinator.com/item?id=5316093
Discussed here: https://news.ycombinator.com/item?id=5361335
If someone is good enough to own you with everything else locked down, they can change any firewall rules completely if they need to, or just tunnel out over an allowed port.
Creating a non-root user then giving them carte blanche sudo rights is similarly odd to me. I'd rather just use root and /etc/nologin (assuming no one else needed a login shell to run).
EDIT: Added paragraph about non-root users.
Agreed it will still stop a script kiddie (do ppl even use that word anymore? Showing my age!) from running random crap on high ports, but thats about it.
You're not going to be 24/7 at the keyboard of the system, there are going to be 0days, or unreleased vulnerabilities, etc.
Egress filtering is important, if you think the chances are large that a user account is compromised a filter table can help. But an even more effective egress filter is one that you run on the router/hardware firewall just upstream from your machines. After all an egress filter on the machine can be disabled by someone that has compromised that machine.
I didn't mean 'intend' in the context of unauthorised use, i was trying to differentiate between a 'single user' ( can't think of a better term, but i'm sure you know what i mean) application server and systems where you purposefully give shell access to other users but want to restrict egress.
The option to block traffic per-application in Linux is no longer included in iptables (I think it was the switch --cmd-owner )
Possibly use a non-standard ssh port, make sure you disable ssh v1 and apply a password policy, allow password logins only for white listed users. Now you should be reasonably safe against brute force attacks and still have a system that is accessible.
If/when you disable root logins through ssh, try to have another way to login as root - maybe a console/kvm switch, prefferably with remote access through a secure network.
Could you explain what's the "easy" way to DoS using fail2ban(or equivalent system)?
Law of unintended consequences strikes again.
The whole article sounds like a lot of fud to me. For example, what benefit is there in creating a new user with sudo privileges instead of using root directly?
Users should run with as few privileges as possible. Logging in as root requires two sets of keys.
Sudo logs all commands. The commands usable when using sudo can be limited, whereas root gives access to everything. Using sudo creates heightened awareness of risks.
Root is a high value target. Giving it a strong password is important, but locking it is better.
Also, when a group of people all need access to the same servers, you don't want everyone SSHing in as root. It's much easier to manage team security when everyone has individual users and sudo rights that can be logged and audited.
In all honesty; the steps in the TO's article are only providing basic security. Pointing the ansible playbooks to your server will further enhance it.
You should also consider changing the SSH port, but that's up to you (in which case you also need to change the firewall settings!)
[0] https://www.debian-administration.org/articles/491
http://www.nsa.gov/ia/mitigation_guidance/security_configura...
A few feedback:
1) Start by the "lower layer"
If it's a physical server, start by reviewing all BIOS options, if it's virtual one you control, start by reviewing the host environment for the guest.
2) Follow from the bottom to the top
Follow by review and secure the boot process. (Grub password, kernel modules loaded, kernel module options, kernel sysctl options.
Follow by review the services started "even before starting nothing".
Review what is accessible in the system to "anybody".
Review what is accessible in the system to each "user". Many times a "user" is going to be listening to the hostile network in a port.
Review what the system is going to do across the time, what is scheduled, which resources are assigned to which users (and their processes), what is going to be registered.
4) Does not limit or scrub any traffic
It's not that we forget about IPv6
It's that such iptables, on any 1 core VPS, is vulnerable to DoS from a single IP (you don't need any distributed attack to put down that). A syn attack to the port 22 could be enough.
In resume:
When giving general security advisories, lets try that people build a secure system if they follow our tips. Not let open things like ping broadcast from our host, or IPv6
That being said, I've read your update, and that is the point that people seeking for "recipes" should get. If they are going to touch ssh or iptables, they need to be ready to explore the documented options (which are many), to validate that the changes are working, to dive into networking, etc.
http://grsecurity.net
Having said that, I'm surprised Debian and Ubuntu don't have better support for grsecurity. If they'd provide up-to-date packages I'd switch over.
Indeed, there's always the risk to lose your private key. That's why it's useful to have several : I have one on my desktop, one on my laptop and one on my mobile. If any get compromised, I only have to delete one public key in authorized keys and generate a new one. That's still open to some attacks, but way less dangerous than simply allowing anyone from anywhere to log in.
That's also better than allowing only a few IPs on port 22, because you'll never now where you will be and what IP your laptop will have when an emergency occurs.
I think you'd be astonished at how weak these are. That's part of the problem: it's hard, and not getting easier, to pick strong passwords.
Personally, I still don't like SSH keys. Yes, they are "more secure" in the sense of preventing others from breaking into your servers. But they are less reliable in a sense that they increase your probability of being completely locked out when shit hits the fan.
Laptop, desktop and mobile phone? If you can't imagine a situation where you will loose access to all three at once, you either live a very different lifestyle than I do or have a poor imagination.
For me, the expected loss (i.e. probability * damage) of loosing access to an important server because of some fire of flood is far greater than the expected loss of someone guessing SSH port, my login, my password and then the root password.
It does. You can encrypt the key and then you have two-factor authentication. You need to have the key and know the password to use it.
Actual two-factor authentication would be requiring proof of access to the secret key and a diffrent login password.
You shouldn't just update, you should update regularly or better yet set up unattended upgrades[1]. Especially for your hobby projects or personal server because odds are that you won't always have the time to act on every security advisory. (Subscribe here[2] to at least hear about them.) Also, if something breaks once in a blue moon, it's not that big a deal.
Fail2ban is fairly heavy and only very recently supports IPv6 (which means the version from your repo may not). You can achieve similar results with something like
but if you disable password logins, no one's going to brute force their way in.[1] https://help.ubuntu.com/community/AutomaticSecurityUpdates
[2] http://www.ubuntu.com/usn/
(Why? Because you didn't read the caution note that upstream sent along to you, that's why. You just woke up to the alert saying that your system is down.)
If you have a reasonable environment, you should have alpha or qa servers that look just like production, except that they aren't visible to the outside. Always test changes there before upgrading on your production machines.
Agreed. Indeed, the parent explicitly shows one possible trade-off:
Especially for your hobby projects or personal server because odds are that you won't always have the time to act on every security advisory.
The limit module is a beautiful thing, since you can also use it to limit log messages generated for events etc., using the same syntax, like so:
Might as well set it automatically and stay protected <shrugs>
I'm not sure how good the Ubuntu LTS relases are, but Debian has always been great at both keeping a stable system up, and keeping the upgrade to new version as painless as possible.
[1] "upgrade" if using apt-get, "safe-upgrade" if using aptitude
So I had the effect right, but the commands mixed up :-)
Ten years ago, I had a Debian server completely wedge itself on an update. glibc was hosed, practically nothing would run, and I couldn't untangle the mess. Had it been unattended, downtime would have been even longer than the hours it took to rebuild the box from scratch.
This was not the first or last time such an event would occur, merely the most severe.
Updates break things. Anyone who claims otherwise is either incredibly lucky, or incredibly inexperienced.
For a highly-available mission-critical infrastructure in 2013, the failure plan is also to rebuild from scratch, because "failure" means redundant and backup systems have exploded. This is most likely to occur when you automatically roll out untested changes to your infrastructure.
In either case, you won't be sleeping tonight. Or possibly tomorrow night.
Recently, we had Bryan Kennedy's "first 5 minutes" [3], linked from Drew Crawford's guide to "NSA proofing" email [4] (pretty good guide to securing a mail server).
[1] https://access.redhat.com/site/documentation/Red_Hat_Enterpr... [2] http://www.nsa.gov/ia/mitigation_guidance/security_configura... [3] http://plusbryan.com/my-first-5-minutes-on-a-server-or-essen... [4] http://sealedabstract.com/code/nsa-proof-your-e-mail-in-2-ho...
http://code.google.com/p/lifs/
Why is it in frontpage?
Most truly competent sysadmins already have full-time employment by a product company or hosting provider. Most so-called sysadmin consultants are former web developers who taught themselves Linux and security, and don't know what they don't know.