You can actually autodetect whether the terminal background is light or dark. For any xterm-compatible terminal, write '\x1b]11;?\x07' to the terminal, and it'll write back a string telling you the foreground color (for instance, '\x1b]11;rgb:0000/0000/0000\x07', which if written back would set the foreground color). If the color matches 'rgb/RRRR/GGGG/BBBB', compute the luminance of that color, and assume a dark background if <0.5 and light otherwise.
Awesome; more tools should do that. Some caveats, though:
* You might not get a response from every terminal, so limit how long you wait.
* If you don't already have echo turned off, turn if off before sending the sequence, because otherwise it'll be visible as though the user typed it.
* You don't know that the color will use the "rgb:RRRR/GGGG/BBBB" format (a terminal can return anything XParseColor can understand); just read the string from the escape to the terminator, look for 'rgb:', and ignore formats you don't understand.
> Have you measured how long typical terminals take to respond?
Arbitrarily long. Consider that a user might run your application over SSH via a high-latency network connection. Better to just handle it asynchronously. Your input loop needs to watch for escape sequences anyway, so watch for that one and process it when or if you see it.
Sadly, that only works for interactive screen-oriented applications, not run-and-exit command-line applications that want to use color.
> Regarding the third point, it might be a good idea to just feed it to XParseColor and process it from there.
That assumes you have libX11 and an X Display available. The former is a heavy dependency for a CLI application, and the latter requires you to connect to the X server.
I'd suggest just manually handling the common case of "rgb:R/G/B" (where each component may use 1-4 digits and requires scaling accordingly), and then deal with anything else if your users actually encounter it in the wild.
Actually, I've improved on this somewhat by splitting configs to 6.5+ vs. older. Corrections welcome. At some point, I will get around to publishing it.
It covers 10x what all the other guides cover in terms of server and application security. It was posted a few weeks ago on HN but didn't make the front-page.
My worry here is that, in posting what seems to be a book, people just won't even do it because we don't have time to do it, unless this is a primary part of their jobs.
If a 10 minute guide gets users 90% of the way, then they're more likely to do it. And that's good enough to cover a majority of automated attacks.
Update: I take it back – they've provided scripts to run this stuff. I will explore these. Thanks for the link.
Agreed - it is a bit long. But if you want to set something up from top to bottom it may be worth it. Cool - I saw the link to Github as well, looks like there is code to do much of what the article outlines. Awesome.
The problem isn't that it's long; the problem is that it's not navigable - there needs to be a table of contents. There's great stuff in here, but it's hard to sort out what I already know how to do from what I can actually use.
Yeah, but it's a PDF. Anyway, why can't you generate a TOC into the HTML version from whatever format you're actually authoring in? Good tooling support should make this not hard.
EDIT: And now the HTML version has a TOC, too! Talk about immediate gratification -- kudos to whoever did that!
Back in the day, The Linux Documentation Project had a trove of hundreds of HOWTOs covering every facet of using Linux. By today's standards they seem like "books", but in reality they were step-by-step instructions for anything you could ever want to do in Linux. No digging through forums, no combing through man pages, no following broken outdated blog posts that didn't explain what you were doing. I find it sad that these are seen as detrimental today.
I feel like arch wiki is that thing today. It's a bit disorganised and things get outdated while nobody's looking, but there's almost anything you'd every want in there.
It seems like there a bunch of articles there but they're more like notes on how to use a tool specifically with Arch, rather than a guide for everyone.
Compare this page http://www.tldp.org/HOWTO/Quota.html to this page https://wiki.archlinux.org/index.php/Disk_quota . TLDP organized HOWTOs like mini-books; tables of content, multiple authors, versioned releases, and of course, you could download them all and search through them by category. And they didn't assume things like the distribution setting up a bunch of the tools and system for you, so you learned how the tools actually worked.
Sure, it's a different format. But you can always dig into man pages or other documentation to learn the details. Arch wiki is great for practical solutions - for example https://wiki.archlinux.org/index.php/HiDPI is the place to go to for hidpi display configuration. TLDP has not been seriously updated since hidpi even came out. Similar to https://wiki.archlinux.org/index.php/Power_management - there's no recent TLDP equivalent.
I appreciate TLDP for what it is and the details they go to. But Arch wiki can easily be read as a FAQ for all modern systems. And it will usually send you to other places for details (sometimes TLDP as well)
Yes. We originally just put this together just as a primer for younger engineers to understand the basic steps. The guide you posted looks really good though. It looks like it covers some of the application side of things as well.
So many people get some of the most basic security wrong. It is great to see people trying to educate instead of just pointing out issues. Thanks for this work.
BCrypt works fine, but I wouldn't say it is "2016-era security practices". It was written in 1999 and hasn't had as much scrutiny as SHA or Blowfish (although it is based on Blowfish).
Regardless, using a salted, multi-pass algorithm will keep everything nicely secured using nearly any hashing algorithm.
Remember the goal is not to crack one user's password using a brute force lookup table, it is to crack everyone's password.
Github recommends 4096 now, for what it's worth. [1]
>Pushing database backups offsite
This is a really bad idea and a good way to get owned. Database backups must be PULLED from the server, not pushed from it. Separately, you also need to test that you can restore from your backups periodically.
There were a couple other things I disagree with, but they're in the realm of personal preference. It's also interesting that this guide uses Linode which is a hosting company known to have had quite egregious security issues in the past. [2]
If a server can push backups, it must necessarily have credentials for and write access to that backup repository. If the server is compromised, that write access could be used to attack the backup repository as well. Pulling backups from a server removes the need for credentials or access to the backup repository.
That is one valid way of solving this problem. Another option is to have a permission system that allows you specify that the server can only append new data, but not delete or overwrite existing files. This is useful if you don't want the added complexity of having a separate "backup management" service, which could easily become a bottleneck if you are backing up a lot of data from multiple servers.
One way to do this is with S3, for example, is to use an IAM role with only the "PutObject" permission, and enable object versioning for the bucket to prevent a compromised server from being able to delete data by overwriting existing files.
It's also easy to do this with Tarsnap. You can split out your host keys with tarsnap-keymgmt into separate read, write, and delete keys. Just take your delete key off of the host and store it somewhere secure.
Or better yet, create the key somewhere else and only upload the keys you want to have on the server. Otherwise there's a chance that the original tarsnap key is left in an unallocated block.
Not an expert here, but this seems to me to be stripping away one layer of security. Kind of like saying that you can leave a key to your house lying around as long as the pantry door is locked - if your database server gets owned, they now have some kind of login to your backup server as opposed to no kind of login.
Maybe the extra layer doesn't add much security, but if it's a simple config change and it does add something, wouldn't it be worth doing?
You could make a similar point about the centralized backup management server - it needs to have "some kind of login" to all your production database server, so of that host is compromised (which might only store encrypted copies of your backup), so are all your database servers if those privileges can be escalated. You could argue that one backup host is easier to secure than a complex system such as AWS, but then I would argue that it would probably be hard to beat the track record of S3/IAM. ;-)
Both approaches have their place. If you're dealing with a large number of hosts and a lot of data, the backup host will quickly become a bottleneck and you're probably better off with the append-only approach. If that's not a concern for you (i.e. you're not running into any bandwidth limits on the backup hosts) and want to deal with operating yet another service to avoid the risk of e.g. IAM privilege escalation, the other solution might be a better fit.
Doesn't entirely fix the problem, but using public/private key auth and a private key created just for the backup cron job, the server can push a backup via scp/ssh to its own user account which on the destination end is chrooted into a particular backup destination directory.
>This is a really bad idea and a good way to get owned. Database backups must be PULLED
Just make sure your database server doesn't have permission to delete backups (e.g. Have it POST a backup via HTTPS). There is nothing wrong with db server initiated backups.
This is still open to abuse in the form of your legitimate backup script being disabled and replaced by one uploading false backups. You wouldn't know anything has happened until you go to restore from one.
Of course, this is an edge case, but I believe setting up a pull-based backup system is still going to be less work than a write-only push system.
I agree that pulling does appear to be more secure, but if someone breaches any of your servers undetected, you're pretty much hosed regardless.
Plus, the Inversoft guide specifically states that backups must be encrypted. I could put my backup ZIPs on a public Github repository and no one would be able to access the user data stored inside it. Therefore, it really doesn't matted if they are pushed or pulled.
If a script or cronjob or whatever on the server has write access to the repo, so does the hypothetical attacker.
I remember seeing a "hacking" website get wiped along with all of its backups about 15 years ago and it left a very strong impression on me regarding this issue.
I don't currently use them, but I've had good luck with Vultr and they haven't had any incidents like Linode AFAIK. There's also AWS, Azure, whatever Google's thing is called, and all the other big ones. I have no opinion on DigitalOcean even though I currently have a VPS with them since I'm not running anything remotely mission critical there.
I'd been using this article for a couple years when I was a Linux server neophyte and now some of these things seem obvious to me. A good article for total noobs.
Why don't they disable root logins with password period and only allow SSH key authentication?
Also if you put a passphase on your SSH key does that mean you have to enter it every time you want to SSH to the server (in order to unlock the key) or does it stay cached on most SSH clients (ssh on mac terminal, putty on windows, etc).
Isn't watching failed logins kind of useless? I think it is more important to see what successful logins were made.
> Isn't watching failed logins kind of useless? I think it is more important to see what successful logins were made.
Are you talking about Fail2Ban? I'm not an expert but my guess is that it's defense in depth. Rate limiting failed logins is useful in the case where you messed up somewhere else and have a service/login that is vulnerable to password guessing.
I would imagine that auditing successful logins is definitely useful but was left out of the guide for brevity.
They do, don't they? Later down there is a section Enforce ssh key logins. However that makes activating the root account even stranger.
The thing with the ssh key passphrase makes not much sense to me. I think this is just about "keep your private key save". On linux, passwords like this (like gpg) get cached by the usual password input clients.
You are right about watching the successful logins first. He writes it is to raise awareness.
Production boxes are not allowed to be plugged into bare internet or DMZ unless they were seasoned and tested on the LAN first, and the LAN allows no external traffic in (stateful firewall). If people on your own LAN are trying to pown you, you got bigger problems to solve before installing another box.
In the old days this was manually moving ethernet cables, now a days this is changing which VLAN the virtual image talks to or if you use something like openstack that implements its own firewall at the virtualization level you allow no external traffic in until the config and testing is done.
Also you need to verify your install media is not powned, which means you need access to the md5sum of the media (and how do you know someone didn't MITM the correct md5sum?) and you need to verify your md5sum program isn't powned which means you need to verify your verification strategy isn't powned which means this gets recursive real quick.
In general a VPS should come up more or less up to date with your SSH key pre installed and no services running except SSH. It may permit password auth but no accounts would have passwords.
I was going to ask the same thing. How do you stop someone from owning a new Linode between the time you start the server and the time you disable password authentication?
Can somebody help me out with this question: The default config for unattended-upgrades seems to not enable reboot even if a reboot would be required to activate the upgrades.
Wouldn't that had made quite a few important upgrades in the last years effectless if they server did never get rebooted?
Yes. You should still keep your eye out on patches. If a big vulnerability gets patched requiring a reboot, you'll want to attend to the upgrades yourself.
Not sure if this is still true, but I've also seen cases where Ubuntu will happily continue to install kernel updates as they come down the pipe, right up until /boot is full of old kernels and ramdisks.
> We don't even have a password for our root user. We'll want to select something random and complex.
So you're taking something secure by default -- no password means no login allowed, and making it less secure. And if you have hundreds of these servers, you'll need to rotate them whenever someone on the team leaves. This is painful.
Simple solution: leave root password blank, don't forget your sudo password. If you can't get in, use grub or a liveCD. Or tie auth to ldap or kerberos so you _can't_ forget. This is one area where Windows has a distinct advantage: AD more or less requires admins to think at the level of network of servers, and provides a baseline set of services always present.
It depends on your VPS, but many give a root password by default.
I do make sure later in the article that `/etc/ssh/sshd_config` does not allow root login:
PermitRootLogin no
But you make a good point that a simple solution is just having no root password at all. If your VPS does have a root login by default, then I believe you can get rid of it with:
sudo usermod -p '!' root
The best part about sharing things like this is getting all sorts of great info and input on things.
So it looks like from the `passwd` man page that `-l` locks it, but doesn't actually remove it completely? I wonder if there are any disadvantages to this.
Locking means it changes the password to a value that nothing could ever hash to (by putting a ! in front of it, for example). Unlocking just removes the !.
The security disadvantage is that if someone gets your /etc/shadow file they can still see the password hash, in case they wanted to try to crack it, but honestly at that point your server is likely compromised anyway.
Not on (eg) BSD. I don't know if this is Linux-only, or more pervasive, but certainly not entirely standard. On Net, Free, Dragon Fly and OpenBSD "-l" mean "update only the local database", versus Kerberos.
Well, sure, but the usermod method I was remarking on isn't standard either, and it established a Linux context for this thread. I'm not sure there's a truly portable way to lock a password across SysV, Linux, and BSD systems.
The _effect_ of what you do (make an "impossible hash") is the way, as far as I know -- whether its vipw, passwd -l, whatever...
Regardless -- I'm not trying to diminish your solution, nor Linux; I elided over the usermod example in the grandparent (didn't recognize it at all), but interestingly, as I look on my NetBSD system,there is a usermod(8), with a -p for already hashed passwords. As I test it though, it rejects '!' as an argument:
The last company I was at was more like this and had admins who were really anti LDAP...because...I have no clue.
You should have centralized auth whenever possible. LDAP servers with fallback, Shibboleth for SSO on web apps (or to implement ADFS integration or SSO with Slack or whatever),
I'd also recommend these steps being combined with Vagrant + your configuration management tools (Ansible, Chef, Puppet).
There are Vagrant + Linode, Vagrant + KVM/libvrt, Vagrant + Digital Ocean, etc. If you combine the two, you can get pretty close between local virtualbox instances and production (you'll run into some issues with each providers base box being a little different, but you can usually accommodate for both your own box and your provider's)
This makes it a little easier to move your architecture from one provider to another.
This part struck me as odd, too, especially when you consider the justification: That if you lose access to your sudo account/password, you must have some other way to get into the system.
Backups should include everything required to rebuild every server in the company. No server should be 'too critical' to wipe and start again if required. So maybe I'm being too much of an armchair warrior here, but the reason I don't like this one is less about the actual security implications, and more because of what it says about the fragility of the overall setup.
>Backups should include everything required to rebuild every server in the company. No server should be 'too critical' to wipe and start again if required.
True. But compare the effort required for, say, solving an accidentally misconfigured /etc/sudoers on a database server. You could argue that would never happen with proper testing, but shit happens and and having a particularly long root password in Keepass is a small price to pay to save that sort of headache.
You still can't login to the system. Root login is disabled as well as password authentication. The root password is just in case you lose your sudo password but it doesn't provide you a way to login if you don't have your key, passphrase to your key, access to the static IP/VPN and device with 2FA.
It also was more robust for me and simpler to configure.
Though I switched from sshguard to fail2ban, because after I started blocking attacks on ssh, script kiddies started to brute fore passwords using other services (SMTP/IMAP/POP3 etc.)
Edit: looks like they added ability to monitor other services, I guess I'll re-evaluate it again.
I can't see any benefit, what am I missing? Put SSH on a port that's not 22 and done, no more mass scanning. The only thing SSHGuard has ever done for me is to lock me out when I was accidentally using the wrong key.
If SSH is on a non-standard port, it is still possible to brute-force access to the server. You will see fewer automated attempts but you are still vulnerable to a motivated attacker who port scans you and finds the SSH port. Such an attacker is less common than automated scans but is more of a threat. With Sshguard, you are no longer vulnerable to this type of attack at all, no matter which port you run SSH on.
If you can make hundreds of login attempts per second, and you can keep at it for days/weeks/years, you can get through some pretty big password dictionaries with lots of variants (e.g. password/p4ssword/passw0rd/p4ssw0rd/etc.).
Technically you don't need the root password, you can always password recovery if you have access to the box. And how exactly did you lock yourself out of every account with sudo? Of course there's always "messed up my ldap or general network settings, can't log in to fix them". There's nothing wrong with setting your root password to a random string and throwing it away, after verifying your sudo works, I guess.
I will admit to being lazy, and with full automation its faster to spawn a new virtual image and let ansible run its course than to do root password recovery where you boot and tell the bootloader to make the init system /bin/sh and hand edit /etc/shadow and /etc/passwd and then reboot again, etc etc. I mean I can set up a new image almost as fast as I can reboot an old image, and I set up images a lot more often than I do password recovery, so...
Scrap the ssh commentary and set up ssh company wide as per stribika plus or minus local modifications:
"On large scale, you'll be better off with a full automated setup using something like Ansible"
At ANY scale you're better off, unless you're experimenting or time isn't money. It'll take longer to add the time to document and test what you're doing by hand than to convince ansible to do it for you. If you don't document or test you're just doomed, so its not like you can avoid that effort. With automation this is like "first two minutes on a server" not ten.
Some people like to drop a .forward in root's homedir sending mail to your sysadmin mailing list or yourself. I THINK but might be wrong that if you do that you don't have to tell logwatch whom to email to, it'll go to root then forward to the right people. More than logwatch assumes root@something.whatever exists as an email address.
You're missing setting up your centralized rsyslog or local equivalent, your munin/nagios/zabbix or local equivalent... I still configure zabbix by hand because I'm old fashioned but its possible to automate that.
NTP is also missing. You can make Kerberos a very sad faced puppy if time isn't synced. And its easy to set up to point to local trusted servers.
(Note, a post that's nothing but complaining still means the linked article is at least 99.9% correct, it is a nicely written wide ranging TODO list)
> And how exactly did you lock yourself out of every account with sudo?
A single typo in /etc/sudoers or any /etc/sudoers.d file will lock you out of all sudo usage. visudo helps with that, but a single mistake (including in a sudoers.d file installed by a configuration management system or package) will lock you out.
Yes sir, and that's why the paranoid sysadmin ssh's in, sudo su's up, THEN runs ansible and tests that it works before unleashing the ansible (or puppet) across the entire network.
Also if you do the "group auth" thing in sudoers then you edit that file approximately once per employment and never touch sudoers again. Of course that abstracts the problem into "I deleted the wheel (or sudo, or ...) group on the ldap server and now I can't sudo up to fix it". And that's why you make snapshot backups on the NAS, so you can roll back the image of the LDAP server (or whatever you use locally) (and edited to add, and don't do something dumb like use the ldap image running on the openstack to authenticate logins into the openstack... that would be painful indeed)
I prefer lazy sysadmin that avoids all that manual ssh/sudo by making puppet/etc validate sudoers before updating it.
You can syntax check with visudo -c.
Thank you for the notes. I'm on the go right now so sorry for the short reply, but I agree with what you've said. This was more of a primer with some hope to quickly explain the basic steps and what they were doing to younger engineers who don't have a ton of experience with it.
RE ssh note: That's the second time someone has linked it to me. It looks really good. I haven't had a chance to read it yet though. Will do.
Yes, I see a lot of reaction around Unity. To be honest, I had to google it to find out what it was. I have always used Ubuntu Gnome, so I had no clue about Unity nor its issues.
Back to the topic here, I thought someone could outline a security checking for an ubuntu desktop to assess how secure the system is -- or maybe making it secure from a clean install.
It's faster (C++ vs Java) and more compatible with the documents I have to open.
It also pleasantly surprised me once:
I received a Powerpoint presentation, and went to a customer meeting, where said presentation had to be presented.
I plugged the HDMI to VGA adapter for the VideoBeam to the laptop, started the presentation, and the presentation was running in the external display, while the laptop display was still showing the normal 'Powerpoint' view. I could load the web browser in the laptop display to check some things while the presentation was still running undisturbed and the speaker and the audience was happily unaware of it.
That level of professional use in software was something I did not come to expect in Ubuntu for any third party software. In fact I don't know if the other Office suites (including MS) have that feature. I guess they do, but still.
It is wonderful when everything just works as intended.
I would like to recommend using just iptables instead of ufw, I had a case on my vps where an update to ufw failed and then the firewall was not working.
With iptables, install iptables-persistent package so they are saved when you do restarts. Do not try to block entire country ip ranges as this slows the machine down substantially.
fail2ban is great, I would recommend looking at some of your system logs to figure out new rules to add.
Seconding the recommendation to avoid ufw. I haven't actually used it or had a problem with it, but if you understand iptables then you don't need it, and if you don't understand iptables, you're better off just learning to use iptables directly so you can handle those cases that ufw doesn't support or clean up after it when it breaks.
I didn't know about iptables-persistent, but it's easy enough to just "iptables-save > /etc/network/iptables" once you're finished changing the config, and "iptables-restore < /etc/network/iptables" in /etc/rc.local. Pretty sure those binaries come with iptables itself, so they should always be available. (I've never dealt with a system that had iptables where they weren't.)
I didn't notice that iptables-persistent actually saves the currently configured rules periodically. That's both kinda neat and a little scary, and I'm not entirely sure I see much value in persisting dynamic rules; it seems like it'd be easy to end up with a long chain of stale rules that way. Still worth knowing about the automatic persistence, though.
It does, but I seem to remember it having trouble cleaning up after itself when abruptly terminated and restarted. It's been a few years, though, so that's probably no longer an issue.
Ferm [1] is wonderful as an iptables frontend. Apart from making it simpler to read, it can avoid a lot of repetition with handy shortcuts such as ability to group things. E.g.:
.. creates rules for each of the ports listed. You can use multiple groups in the same statement as well (lets say all the services above for some crazy reason also listened to udp - you'd just replace "tcp" with "(tcp udp)").
Being able to set variables is also fantastic. E.g.:
Also, FireHOL [1] is an interesting option. It also has a DSL for managing firewall rules. The custom service port definition is a little weird, but overall I like it better than ferm.
Fail2ban? sshguard? unnecessary.
Just disable ssh passwd auth (which generally is a good idea) -> done/done
If you don't like lognoise from ssh scanners (even if you disable passwd auth), move your sshd port to some random high port and make note of it in your ~/.ssh/config
Generally: if in doubt, take the more simple and elegant solution to a problem.
I agree that fail2ban for ssh seems unnecessary. But, it also provides monitoring for other services like http and common exploits.
I'd be interested in learning from the community if fail2ban adds much value. As I've looked into the service, it seems like simply running the latest security patches obviates the need for fail2ban.
I run a web server with some 50+ Wordpress installs on it. You better believe Fail2Ban is necessary. Without it all resources would be consumed by brute force attacks. If someone knows of a better way I would like to hear about it.
I too am curious to know what the consensus is. People seem to fall into one of two camps: 1. it's unnecessary if you disable password login or 2. it's an easy thing to add as yet another safety net on top of sane service configuration and firewall rules.
I've taken some flak here for being in the #2 camp so I'm loathe to expose myself once again, but my thinking is that even with password authentication disabled, sshd is still vulnerable to DoS attacks. From the public internet or from other machines in my VPS provider's datacenter. The ssh and recidive fail2ban filters add some minimal defense against that.
If all you're worried about is keeping the low-level noise out of your logs and discouraging waste of resources on brute-force attacks (which won't work because of course password auth is disabled), I think it's sufficient to use the built-in iptables rate-limit feature on SYN packets.
UFW has a feature that will implement this with a limit of 6 SYN packets in 30 seconds from individual IPs.
ufw limit ssh/tcp
This may not be ideal for some services like http without persistent connections, and if fail2ban provides extra exploit protection it could be worthwhile.
This is a much better answer than, "If you're using fail2ban for that reason, you don't know what you're doing," as someone said to me recently, so thank you. :-) I'll explore replacing fail2ban with this mechanism.
But those ports are more or less supposed to be available to the general public. You keep those where they're expected to be. Any other port that "does not concern you, move along" is fair game IMHO.
Standards exist so you know where to find the things you're supposed to find, and how to talk to them.
I always worry that adding 2FA could make your machine inaccessible if anything happens to google-authenticator in this case. Maybe it's a little bit of paranoia but I don't like the idea of giving control over my ability to log into my server.
google-authenticator is a fancy name for an algorithm described in RFC 6238. It does not talk to servers or require any network access. Google is not involved in any way; they just made the algorithm popular and branded it.
I didn't read the question as "what if Google discontinues authenticator" but more as "what if I use Google authenticator on my phone and it gets stolen and i can't log in any more"
It'd be perfectly reasonable if libpam-google-authenticator relied on Google's infrastructure, but despite the infelicitous name, it does not; it just implements the server side of TOTP. The authentication flow is identical to any other correct TOTP implementation, and you can use any compatible client; no integration with Google services or infrastructure is required. (In fact, I don't think it's even possible.)
Speaking of TOTP without Google, if you use iOS and find the Google Authenticator app unsatisfactory, try Authy. It's good stuff, and well worth a few bucks.
While I'm not in the market for an OTP client, this is the network tools app I've been looking for since I bought my first iPod Touch. If you have remote beer-buying support, point me to it.
I moved most of my 2FA/TOTP stuff to 1Password. Works great on the Apple watch, and it's one less app (I tried Authy, but I have this thing against using too many apps)
I've been thinking about starting to use a password manager (as I should've done long ago), and 1Password looks pretty good. It's pricey enough that I'd like to hear some pros and cons before I pull the trigger, though. How's it worked out for you? Where are the pain points?
FWIW, the desktop app has a free trial, and on top of that, the developer offers a 30-day money-back guarantee if you buy direct from their store. (Not sure about the mobile app, haven't used that.)
I use the desktop app every day and am generally happy with it. It has many thoughtful touches, like automatically clearing the clipboard a minute or so after you copy a password into it, so that you don't later paste your password into some other text field by accident.
I suppose that's true, but in the field where I work (social science), servers are mostly spun up to scratch an immediate and idiosyncratic itch, so configuration tends to happen organically.
I agree that's probably not a good idea, and learning Pupchefsible is well worth the effort. In the meantime, though, there's at least some degree of reproducibility with etckeeper.
I do this stuff professionally, and I've learned the hard way that you either have a reproducible environment or you don't. etckeeper isn't reproducible. Actually rolling back with something like etckeeper is much, much more likely to break something (by deleting a config file used by a newer service, say) than to save you. If it did something like separate branches for each service or component within /etc I might be more sympathetic...but at that point you have half of a CM system already and might as well just go the rest of the way.
If you need reproducibility without a CM framework, keep backups of your machines.
I would say the same reason we teach web developers what a linked list is. They might never use it (watch me eat my own boot here) but it's a fundamental principle upon which a lot of the tools they use are built.
Not really. It's more like `./provision.sh your-new-host.example.org`
(Because why in the world would anyone want to type in things by hand? Laziness for any repetetive manual labor is a greatest virtue of any good sysadmin.)
Because Linux systems can vary widely. Even the same distro can have subtle - or even significant - differences from one hosting provider to another. So writing a shell script that was robust enough to "just work" would not only be complex, it would also likely suffer from readability issues (as so many complex shell scripts do) and thus many sysadmins might not want to run something that can't audit - which is quite sensible really. In fact you cannot even guarantee that Bash will be installed, that's just how annoyingly complicated this job can be.
As a side note, it's also sometimes better having a tutorial full of manual steps as that helps educate the "sysadmin" regarding best practices and some of the basics of Linux administration (if they weren't already familiar). That experience can be just as valuable as hardening the server itself.
They could be, or you could take the route you won't regret later and use a configuration management system. Deutsche Telekom has open-sourced system-hardening cookbooks for Chef, Puppet, and Ansible.
You are supposed to understand the implications of all of these steps. If by accident you ever lock yourself out, you will need to know how you set it up. If you just recommend running a script, most people will not learn what they are doing.
Yes. Doing this 'manually' step by step lets you choose which things are really necessary, your configuration might be different, and forces you to learn WHAT it's doing rather than just running a script. I am not a huge fan of Arch Linux (prefer debian unstable), but this is the same principle as the Arch Linux installation process which doesn't have a monolithic installer but forces you to do the process manually step-by-step.
Because newcomers would then copy-paste the script without really understanding what's going on, and that lack of understanding will come back to bite when something breaks. Teach a man how to fish rather than giving him the fish and all that. Writing a bash script as a next step based on OP's article would be easy once you know what you are doing.
You should really automate this process if you create more than one server, ever. It's fairly easy to get the basics, and if you don't automate it, I guarantee you'll miss one or more steps in the setup.
It would run through a series of questions about your use case to build a security policy, and then edit config files for you.
The problem is that it needs to be aware of all the different flavors of Linux it might be running on, so it's naturally fragile and requires lots of maintenance. Sadly, it hasn't been updated in a few years.
Hardening (along with any other server setup/configuration) should be implemented using some idempotent configuration management software (Chef, Puppet, Ansible, etc). It's 2016, there's no need to configure servers manually.
Other than to learn how, which is also why I'd provide text on how and now a bash script to just do it. Not everyone's needs are the exact same, and it's better to learn how to fish than be handed a line with one hooked. Maybe I have a strong reason not to go SSH keys only? I should at least understand how they work and why they're important so I can make the decision, wrong as many people think it may be.
I mention Ansible and other tools in the article. The idea, as others have mentioned, is to teach a man to fish. You're much better off with having an automated way to perform this, but you shouldn't ever run anything automated that you don't understand.
We'll release an Ansible Playbook over the next week or so that follows these steps.
I'm a security guy, so it would be embarrassing and possibly bad for my career if any of my servers got hacked, but I actually don't set up servers that often--it's not part of my job. These Web 2.0 configuration management solutions change pretty fast and don't care about reverse compatibility. So between my infrequent setups my configuration scripts pretty much always break.
Contrast this with bash, which cares a whole lot about reverse compatibility, and it's a no brainer. I've got scripts where the only modifications I've made since 2005 were to add functionality or increase security, never to fix existing functionality that was broken by a change to the system. I'll take that over Chef or Ansible (both tools I've used) any day.
Which is why your hardening code (even if it's bash scripts) should have tests, using something like Test Kitchen. It will provision a VM (or cloud server, or docker container), execute your hardening scripts against them, and then run your integration tests to verify that the machine is hardened to your specifications.
Why do people install fail2ban then disable password based authentication entirely? I legitimately don't understand the purpose.
Also, they complain about log spam (from failed SSH attempts) this is one reason to move SSH to a different port. It does NOT increase security, but it DOES reduce log spam from bots trying for easy targets.
I always change the ssh port to something other than 22. It has always seemed to work well for me for most automated attacks. Perhaps this is not advisable for some reason?
fail2ban installed = less logspam, and never having to adjust default port with whatever tool is ssh'ing
moving ssh port = less logspam, but now you have to specify your ssh port in every tool that uses it, and it's one more thing that other staff have to remember
Have you tried it both ways? some scripted attacks will try 22 find that it is open then spam it with attacks. If it's not open they will have to scan your ports then spam you with attacks
Well for one, every command you run has root privileges (instead of requiring sudo). Every process you spawn has root privileges. You're safer keeping your privileges limited and sudoing when required. Also you lose your real-life audit log if multiple people log in as root.
But when I tried using a regular user to run anything on a server - it still requires me to do sudo, so the process will anyway have root privileges, no? So I end up typing sudo most of the times becauase most of processes do not work without root priveleges
297 comments
[ 4.7 ms ] story [ 270 ms ] thread* You might not get a response from every terminal, so limit how long you wait.
* If you don't already have echo turned off, turn if off before sending the sequence, because otherwise it'll be visible as though the user typed it.
* You don't know that the color will use the "rgb:RRRR/GGGG/BBBB" format (a terminal can return anything XParseColor can understand); just read the string from the escape to the terminator, look for 'rgb:', and ignore formats you don't understand.
* To calculate whether a color is "light" or "dark", see https://en.wikipedia.org/wiki/Luma_%28video%29:
Regarding the third point, it might be a good idea to just feed it to XParseColor and process it from there.
Arbitrarily long. Consider that a user might run your application over SSH via a high-latency network connection. Better to just handle it asynchronously. Your input loop needs to watch for escape sequences anyway, so watch for that one and process it when or if you see it.
Sadly, that only works for interactive screen-oriented applications, not run-and-exit command-line applications that want to use color.
> Regarding the third point, it might be a good idea to just feed it to XParseColor and process it from there.
That assumes you have libX11 and an X Display available. The former is a heavy dependency for a CLI application, and the latter requires you to connect to the X server.
I'd suggest just manually handling the common case of "rgb:R/G/B" (where each component may use 1-4 digits and requires scaling accordingly), and then deal with anything else if your users actually encounter it in the wild.
I was speaking of this regarding screen-oriented termbox/curses applications, not run-and-exit applications.
"My goal with this post here is to make NSA analysts sad."
I haven't seen this article before. It looks quite good. Thanks for posting it.
Configs
OpenSSH 6.5+ Server
UsePrivilegeSeparation sandbox KexAlgorithms curve25519-sha256@libssh.org,diffie-hellman-group14-sha1 Ciphers chacha20-poly1305@openssh.com,aes256-gcm@openssh.com,aes128-gcm@openssh.com,aes256-ctr,aes128-ctr MACs hmac-sha2-512-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512,hmac-sha2-256
OpenSSH Server Legacy
#UsePrivilegeSeparation yes KexAlgorithms diffie-hellman-group14-sha1 Ciphers aes256-ctr,aes128-ctr MACs hmac-sha2-512,hmac-sha2-256
OpenSSH 6.5+ Client
UseRoaming no IdentitiesOnly yes KexAlgorithms curve25519-sha256@libssh.org,diffie-hellman-group14-sha1,diffie-hellman-group-exchange-sha256,diffie-hellman-group-exchange-sha1 HostKeyAlgorithms ssh-ed25519-cert-v01@openssh.com,ssh-ed25519,ssh-rsa-cert-v01@openssh.com,ssh-rsa Ciphers chacha20-poly1305@openssh.com,aes256-gcm@openssh.com,aes128-gcm@openssh.com,aes256-ctr,aes128-ctr MACs hmac-sha2-512-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha1-etm@openssh.com,hmac-sha2-512,hmac-sha2-256,hmac-sha1
Host * IdentityFile ~/.ssh/id_ed25519 IdentityFile ~/.ssh/id_rsa HashKnownHosts yes VisualHostKey yes VerifyHostKeyDNS ask AddressFamily inet ForwardX11 no ForwardX11Trusted no
OpenSSH Client Legacy
UseRoaming no IdentitiesOnly yes KexAlgorithms diffie-hellman-group14-sha1 HostKeyAlgorithms ssh-rsa-cert-v01@openssh.com,ssh-rsa Ciphers aes256-ctr,aes128-ctr MACs hmac-sha2-512-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha1-etm@openssh.com,hmac-sha2-512,hmac-sha2-256,hmac-sha1
Host * IdentityFile ~/.ssh/id_rsa HashKnownHosts yes VisualHostKey yes VerifyHostKeyDNS ask AddressFamily inet ForwardX11 no ForwardX11Trusted no
https://www.inversoft.com/guides/2016-guide-to-user-data-sec...
It covers 10x what all the other guides cover in terms of server and application security. It was posted a few weeks ago on HN but didn't make the front-page.
If a 10 minute guide gets users 90% of the way, then they're more likely to do it. And that's good enough to cover a majority of automated attacks.
Update: I take it back – they've provided scripts to run this stuff. I will explore these. Thanks for the link.
https://github.com/inversoft/passport-js-example
It uses Ember, Node.js, Express, Sequelize, MySQL and Passport User Database (https://www.inversoft.com/products/user-database-sso).
EDIT: And now the HTML version has a TOC, too! Talk about immediate gratification -- kudos to whoever did that!
E.g. Apache 1, BIND, sendmail, cgi-bin, ifconfig, etc.
Compare this page http://www.tldp.org/HOWTO/Quota.html to this page https://wiki.archlinux.org/index.php/Disk_quota . TLDP organized HOWTOs like mini-books; tables of content, multiple authors, versioned releases, and of course, you could download them all and search through them by category. And they didn't assume things like the distribution setting up a bunch of the tools and system for you, so you learned how the tools actually worked.
I appreciate TLDP for what it is and the details they go to. But Arch wiki can easily be read as a FAQ for all modern systems. And it will usually send you to other places for details (sometimes TLDP as well)
https://www.digitalocean.com/community/tutorials/7-security-...
https://github.com/FallibleInc/security-guide-for-developers... (work in progress)
> Passwords should always be hashed using a strong, one-way hash algorithm. [...] hashed with an algorithm like SHA-256 7 times.
If they had simply written "just use bcrypt" they'd be much closer to 2016-era security practices.
Regardless, using a salted, multi-pass algorithm will keep everything nicely secured using nearly any hashing algorithm.
Remember the goal is not to crack one user's password using a brute force lookup table, it is to crack everyone's password.
And it is designed to prevent what your last sentence says.
Github recommends 4096 now, for what it's worth. [1]
>Pushing database backups offsite
This is a really bad idea and a good way to get owned. Database backups must be PULLED from the server, not pushed from it. Separately, you also need to test that you can restore from your backups periodically.
There were a couple other things I disagree with, but they're in the realm of personal preference. It's also interesting that this guide uses Linode which is a hosting company known to have had quite egregious security issues in the past. [2]
[1] https://help.github.com/articles/generating-a-new-ssh-key-an...
[2] https://www.google.com/#q=linode+hack
One way to do this is with S3, for example, is to use an IAM role with only the "PutObject" permission, and enable object versioning for the bucket to prevent a compromised server from being able to delete data by overwriting existing files.
Maybe the extra layer doesn't add much security, but if it's a simple config change and it does add something, wouldn't it be worth doing?
You could make a similar point about the centralized backup management server - it needs to have "some kind of login" to all your production database server, so of that host is compromised (which might only store encrypted copies of your backup), so are all your database servers if those privileges can be escalated. You could argue that one backup host is easier to secure than a complex system such as AWS, but then I would argue that it would probably be hard to beat the track record of S3/IAM. ;-)
Both approaches have their place. If you're dealing with a large number of hosts and a lot of data, the backup host will quickly become a bottleneck and you're probably better off with the append-only approach. If that's not a concern for you (i.e. you're not running into any bandwidth limits on the backup hosts) and want to deal with operating yet another service to avoid the risk of e.g. IAM privilege escalation, the other solution might be a better fit.
Just make sure your database server doesn't have permission to delete backups (e.g. Have it POST a backup via HTTPS). There is nothing wrong with db server initiated backups.
Of course, this is an edge case, but I believe setting up a pull-based backup system is still going to be less work than a write-only push system.
Of course, your servers shouldn't be SSHing to your backup servers, but that goes both ways.
Plus, the Inversoft guide specifically states that backups must be encrypted. I could put my backup ZIPs on a public Github repository and no one would be able to access the user data stored inside it. Therefore, it really doesn't matted if they are pushed or pulled.
I remember seeing a "hacking" website get wiped along with all of its backups about 15 years ago and it left a very strong impression on me regarding this issue.
It is sometimes more secure to push because it requires no inbound connections or authorizations the live machine.
Also if you put a passphase on your SSH key does that mean you have to enter it every time you want to SSH to the server (in order to unlock the key) or does it stay cached on most SSH clients (ssh on mac terminal, putty on windows, etc).
Isn't watching failed logins kind of useless? I think it is more important to see what successful logins were made.
(see https://help.github.com/articles/working-with-ssh-key-passph... )
Are you talking about Fail2Ban? I'm not an expert but my guess is that it's defense in depth. Rate limiting failed logins is useful in the case where you messed up somewhere else and have a service/login that is vulnerable to password guessing.
I would imagine that auditing successful logins is definitely useful but was left out of the guide for brevity.
The thing with the ssh key passphrase makes not much sense to me. I think this is just about "keep your private key save". On linux, passwords like this (like gpg) get cached by the usual password input clients.
You are right about watching the successful logins first. He writes it is to raise awareness.
Also, good to have in case sudoers file is corrupted, e.g. via bad edit, typo, etc..
In the old days this was manually moving ethernet cables, now a days this is changing which VLAN the virtual image talks to or if you use something like openstack that implements its own firewall at the virtualization level you allow no external traffic in until the config and testing is done.
Also you need to verify your install media is not powned, which means you need access to the md5sum of the media (and how do you know someone didn't MITM the correct md5sum?) and you need to verify your md5sum program isn't powned which means you need to verify your verification strategy isn't powned which means this gets recursive real quick.
So you're taking something secure by default -- no password means no login allowed, and making it less secure. And if you have hundreds of these servers, you'll need to rotate them whenever someone on the team leaves. This is painful.
Simple solution: leave root password blank, don't forget your sudo password. If you can't get in, use grub or a liveCD. Or tie auth to ldap or kerberos so you _can't_ forget. This is one area where Windows has a distinct advantage: AD more or less requires admins to think at the level of network of servers, and provides a baseline set of services always present.
I do make sure later in the article that `/etc/ssh/sshd_config` does not allow root login:
But you make a good point that a simple solution is just having no root password at all. If your VPS does have a root login by default, then I believe you can get rid of it with: The best part about sharing things like this is getting all sorts of great info and input on things.The security disadvantage is that if someone gets your /etc/shadow file they can still see the password hash, in case they wanted to try to crack it, but honestly at that point your server is likely compromised anyway.
Regardless -- I'm not trying to diminish your solution, nor Linux; I elided over the usermod example in the grandparent (didn't recognize it at all), but interestingly, as I look on my NetBSD system,there is a usermod(8), with a -p for already hashed passwords. As I test it though, it rejects '!' as an argument:
You should have centralized auth whenever possible. LDAP servers with fallback, Shibboleth for SSO on web apps (or to implement ADFS integration or SSO with Slack or whatever),
I'd also recommend these steps being combined with Vagrant + your configuration management tools (Ansible, Chef, Puppet).
There are Vagrant + Linode, Vagrant + KVM/libvrt, Vagrant + Digital Ocean, etc. If you combine the two, you can get pretty close between local virtualbox instances and production (you'll run into some issues with each providers base box being a little different, but you can usually accommodate for both your own box and your provider's)
This makes it a little easier to move your architecture from one provider to another.
Backups should include everything required to rebuild every server in the company. No server should be 'too critical' to wipe and start again if required. So maybe I'm being too much of an armchair warrior here, but the reason I don't like this one is less about the actual security implications, and more because of what it says about the fragility of the overall setup.
True. But compare the effort required for, say, solving an accidentally misconfigured /etc/sudoers on a database server. You could argue that would never happen with proper testing, but shit happens and and having a particularly long root password in Keepass is a small price to pay to save that sort of headache.
I really think this should be installed by default on distros like Ubuntu.
[1] http://www.sshguard.net/
Though I switched from sshguard to fail2ban, because after I started blocking attacks on ssh, script kiddies started to brute fore passwords using other services (SMTP/IMAP/POP3 etc.)
Edit: looks like they added ability to monitor other services, I guess I'll re-evaluate it again.
I will admit to being lazy, and with full automation its faster to spawn a new virtual image and let ansible run its course than to do root password recovery where you boot and tell the bootloader to make the init system /bin/sh and hand edit /etc/shadow and /etc/passwd and then reboot again, etc etc. I mean I can set up a new image almost as fast as I can reboot an old image, and I set up images a lot more often than I do password recovery, so...
Scrap the ssh commentary and set up ssh company wide as per stribika plus or minus local modifications:
https://stribika.github.io/2015/01/04/secure-secure-shell.ht...
"On large scale, you'll be better off with a full automated setup using something like Ansible"
At ANY scale you're better off, unless you're experimenting or time isn't money. It'll take longer to add the time to document and test what you're doing by hand than to convince ansible to do it for you. If you don't document or test you're just doomed, so its not like you can avoid that effort. With automation this is like "first two minutes on a server" not ten.
Some people like to drop a .forward in root's homedir sending mail to your sysadmin mailing list or yourself. I THINK but might be wrong that if you do that you don't have to tell logwatch whom to email to, it'll go to root then forward to the right people. More than logwatch assumes root@something.whatever exists as an email address.
You're missing setting up your centralized rsyslog or local equivalent, your munin/nagios/zabbix or local equivalent... I still configure zabbix by hand because I'm old fashioned but its possible to automate that.
NTP is also missing. You can make Kerberos a very sad faced puppy if time isn't synced. And its easy to set up to point to local trusted servers.
(Note, a post that's nothing but complaining still means the linked article is at least 99.9% correct, it is a nicely written wide ranging TODO list)
A single typo in /etc/sudoers or any /etc/sudoers.d file will lock you out of all sudo usage. visudo helps with that, but a single mistake (including in a sudoers.d file installed by a configuration management system or package) will lock you out.
Also if you do the "group auth" thing in sudoers then you edit that file approximately once per employment and never touch sudoers again. Of course that abstracts the problem into "I deleted the wheel (or sudo, or ...) group on the ldap server and now I can't sudo up to fix it". And that's why you make snapshot backups on the NAS, so you can roll back the image of the LDAP server (or whatever you use locally) (and edited to add, and don't do something dumb like use the ldap image running on the openstack to authenticate logins into the openstack... that would be painful indeed)
RE ssh note: That's the second time someone has linked it to me. It looks really good. I haven't had a chance to read it yet though. Will do.
Should take less than 10 minutes and be way faster to use :)
(I actually use openbox and fbpanel, but WindowMaker is just too great to forget about)
In my case: Change the Switch workspace keys from Ctrl+Alt+Arrow keys to Super+ Arrow keys. Remove LibreOffice, install WPS Office. Remove Transmission, install Deluge. Install indicator-multiload, indicator-sound-switcher. Install Kodi. Install Steam.
A lot more stuff, but I have not written it down. =)
Back to the topic here, I thought someone could outline a security checking for an ubuntu desktop to assess how secure the system is -- or maybe making it secure from a clean install.
Edit: grammar
It also pleasantly surprised me once:
I received a Powerpoint presentation, and went to a customer meeting, where said presentation had to be presented.
I plugged the HDMI to VGA adapter for the VideoBeam to the laptop, started the presentation, and the presentation was running in the external display, while the laptop display was still showing the normal 'Powerpoint' view. I could load the web browser in the laptop display to check some things while the presentation was still running undisturbed and the speaker and the audience was happily unaware of it.
That level of professional use in software was something I did not come to expect in Ubuntu for any third party software. In fact I don't know if the other Office suites (including MS) have that feature. I guess they do, but still.
It is wonderful when everything just works as intended.
I would like to recommend using just iptables instead of ufw, I had a case on my vps where an update to ufw failed and then the firewall was not working.
With iptables, install iptables-persistent package so they are saved when you do restarts. Do not try to block entire country ip ranges as this slows the machine down substantially.
fail2ban is great, I would recommend looking at some of your system logs to figure out new rules to add.
I didn't know about iptables-persistent, but it's easy enough to just "iptables-save > /etc/network/iptables" once you're finished changing the config, and "iptables-restore < /etc/network/iptables" in /etc/rc.local. Pretty sure those binaries come with iptables itself, so they should always be available. (I've never dealt with a system that had iptables where they weren't.)
Or is the issue that fail2ban et al don't remove stale rules?
Being able to set variables is also fantastic. E.g.:
[1] https://github.com/MaxKellermann/ferm[1] https://firehol.org/
If you don't like lognoise from ssh scanners (even if you disable passwd auth), move your sshd port to some random high port and make note of it in your ~/.ssh/config
Generally: if in doubt, take the more simple and elegant solution to a problem.
I'd be interested in learning from the community if fail2ban adds much value. As I've looked into the service, it seems like simply running the latest security patches obviates the need for fail2ban.
Parsing a url and returning 404 from nginx is cheap and scalable, and allows through legitimate traffic that may be sharing an IP (such as TOR).
I cannot believe that Wordpress still ships without basic rate limiting on its login form.
I've taken some flak here for being in the #2 camp so I'm loathe to expose myself once again, but my thinking is that even with password authentication disabled, sshd is still vulnerable to DoS attacks. From the public internet or from other machines in my VPS provider's datacenter. The ssh and recidive fail2ban filters add some minimal defense against that.
UFW has a feature that will implement this with a limit of 6 SYN packets in 30 seconds from individual IPs.
This may not be ideal for some services like http without persistent connections, and if fail2ban provides extra exploit protection it could be worthwhile.Sick of seeing this idiotic advice. Standard ports are standard for a reason. Perhaps one should also change their web and mail ports for "security".
> If you don't like lognoise from ssh scanners [...], move your sshd port [...]
It’s not about security as you claim.
Standards exist so you know where to find the things you're supposed to find, and how to talk to them.
Speaking of TOTP without Google, if you use iOS and find the Google Authenticator app unsatisfactory, try Authy. It's good stuff, and well worth a few bucks.
OTP Auth is another excellent, free OTP client.
Edit: clarified both are free.
no beer required :-)
I use the desktop app every day and am generally happy with it. It has many thoughtful touches, like automatically clearing the clipboard a minute or so after you copy a password into it, so that you don't later paste your password into some other text field by accident.
It's of course open source.
apt-get install etckeeper && cd /etc && etckeeper init
Keeps your /etc under version control so you know what kinds of configuration changes you've perpetrated.
Manually modifying servers was never a good idea; it's worse now, even with tools like this.
I agree that's probably not a good idea, and learning Pupchefsible is well worth the effort. In the meantime, though, there's at least some degree of reproducibility with etckeeper.
If you need reproducibility without a CM framework, keep backups of your machines.
It's good to know what's going on under the hood.
It's one of those cases where a tutorial explaining every change you're making and why you're making it really is pretty important.
(Because why in the world would anyone want to type in things by hand? Laziness for any repetetive manual labor is a greatest virtue of any good sysadmin.)
As a side note, it's also sometimes better having a tutorial full of manual steps as that helps educate the "sysadmin" regarding best practices and some of the basics of Linux administration (if they weren't already familiar). That experience can be just as valuable as hardening the server itself.
You should really automate this process if you create more than one server, ever. It's fairly easy to get the basics, and if you don't automate it, I guarantee you'll miss one or more steps in the setup.
That said, if you do such things at scale, you'll likely have automated provisioning and configuration management systems in place anyway.
For example you could have images that have the static parts already pre-configured, and something like cloudinit for the ssh keys and/or passwords.
Or you provision the systems with foreman, and then use puppet for configuration management.
It would run through a series of questions about your use case to build a security policy, and then edit config files for you.
The problem is that it needs to be aware of all the different flavors of Linux it might be running on, so it's naturally fragile and requires lots of maintenance. Sadly, it hasn't been updated in a few years.
Other than to learn how, which is also why I'd provide text on how and now a bash script to just do it. Not everyone's needs are the exact same, and it's better to learn how to fish than be handed a line with one hooked. Maybe I have a strong reason not to go SSH keys only? I should at least understand how they work and why they're important so I can make the decision, wrong as many people think it may be.
Ideally the server comes up and applies chef, doing the needful to secure it. Hand-cooking a server is extremely painful.
We'll release an Ansible Playbook over the next week or so that follows these steps.
I'm a security guy, so it would be embarrassing and possibly bad for my career if any of my servers got hacked, but I actually don't set up servers that often--it's not part of my job. These Web 2.0 configuration management solutions change pretty fast and don't care about reverse compatibility. So between my infrequent setups my configuration scripts pretty much always break.
Contrast this with bash, which cares a whole lot about reverse compatibility, and it's a no brainer. I've got scripts where the only modifications I've made since 2005 were to add functionality or increase security, never to fix existing functionality that was broken by a change to the system. I'll take that over Chef or Ansible (both tools I've used) any day.
Here is Ansible playbook I recently created https://github.com/chhantyal/5minutes
I think scripting all of this is great - but also agree that it is important to understand what is being performed.
One could combine that with something like rundeck where you run apt-get upgrade.
Also, they complain about log spam (from failed SSH attempts) this is one reason to move SSH to a different port. It does NOT increase security, but it DOES reduce log spam from bots trying for easy targets.
Here is a good summary of the different options you have with ssh and choosing a port:
http://serverfault.com/questions/619898/should-i-change-the-...
moving ssh port = less logspam, but now you have to specify your ssh port in every tool that uses it, and it's one more thing that other staff have to remember
Can someone explain me, let's say I disabled password logins and only allow login via a key, what are potential downsides of logging in as a root?