Unrelated to this disclosure, I have acquired quite a lot of Fail2Ban attack reports generated by hosts around the world. I put together an initial set of charts showing attacks by country; I was wondering if anyone had any ideas for interesting things that should be done to analyze/visualize the data.
The data set contains the IP addresses for attacker and victim, the date, and the service name for each of a couple million attack reports.
Do you know where maps between IP addresses and ASNs/netblocks might be available for download? I can't immediately find this on IANA's website or on those of the RIRs, and I think it'd be too much to grab from whois.
Arin will allow you to download portions of their database for research purposes, but it requires an account and it looks like there's an approval process.
You wouldn't have to query every IP, you'd just have to query IPs not covered by a result range from a previous lookup.
I'd be willing to help on this; I have a whois module I wrote for another spam tool, I should be able to pretty easily adapt it for this.
I think that change over time could be pretty cool, especially if the host locations can be resolved down to the city or province. I'll check out D3.js.
He doesn't, he just created the "fail2ban@mail.com" email account there. So all those fail2ban installations with the default configuration send email to him.
«DShield is a community-based collaborative firewall log correlation system. It receives logs from volunteers world wide and uses them to analyze attack trends.»
Fail2ban is fundamentally a wrong answer to the problem. If you're taking the time to install such things, you should instead either be turning off password authentication (relying only on keys) or shorewall (to default deny ssh access except to authorized subnets) or both.
Waiting for multiple authentication failures to mark a host/net as "bad" is fundamentally a bad idea. Stop using passwords.
I've used Fail2Ban even with SSH password authentication turned off. It's still helpful for preventing huge error logs full of pointless password guesses.
It's also useful for applications where you have to use passwords or where you need to allow anonymous requests but you don't want attackers using up all of your resources. VoIP (public-facing and handset-facing SIP) servers are my example of both of those situations.
Except when programs are designed for SSH to communicate on port 22. Security by obscurity is not good practice but other than that, yes, using a non standard SSH port will greatly reduce random drive by attempts.
Unless the majority of ssh brute force attacks iterate over every port value, using a non-standard port would still have value in reducing the volume of attacks.
I'm in no danger of ssh ever being brute'd, but having logs fill up with piles of attempts is annoying and some hosts can be very persistent.
In my case, I also use it to prevent other abuse: bans for ssh abuse protect other hosted services, and fail2ban has been modified to act on other forms of abuse too (like spam).
I'm not sure why this comment is getting downvoted, especially since the users who replied to this comment correctly understand that Fail2Ban is a log prettifier and use it as intended.
It is possible to use brute-force via timing attacks to determine whether accounts exists on hosts, and then either focus different attacks on those accounts or spam them.
It is amazing how much this needs to be repeated, but security is not an absolute. Fail2ban adds another layer of security, thus it is a good thing.
I also use log monitoring and banning to ban http connections to my web server (driven by scanning apache logs), as well as SMTP connections (scanning Exim's rejectlog).
The rationale "do not use password authentication" does not apply to unauthenticated HTTP and SMTP requests.
we do that at work. Found it to painful to make fail2ban to watch to hundreds of vhost log files so we just sniff the requests with ngrep and log them to a file fail2ban watches
that's assuming your use case for fail2ban is only blocking failed ssh login attempts. it's pretty handy for blocking malware scanners, x number of failed logins to wordpress/drupal/etc, and all that jazz.
How do you manage ssh keys once deployed? From personal experience using both systems, my thinking is that once you start deploying ssh keys, in a larger user/server environment, you have a massive rats nest. There is little to no life cycle management.
Fail2ban works and should be deployed regardless of whether you are using usernames/passwords or keys. I just checked one of our bastion hosts, and there are 384 banned IPs (on a rolling 30 day basis). This info is useful, and can be fed into other systems, like edge network devices.
That's one weird article (or maybe people really do that... I would hope they don't):
> mesh of key-based access is so dense that it is highly likely that an attack can spread to nearly all servers in an organization
Why would you ever do that? Keep the keys where the users are. There is no mesh unless you distribute the private keys and you should never do that in the first place. ssh-agent can be used if you really need to have multiple hops.
> odds are the virus will infect nearly all servers [...] including disaster recovery and backup systems that are typically also managed using such keys
Why would you store private keys to your DR/backup servers all over the place? Backup is safer if it works via pull.
There are problems with SSH key management, but the situation they describe looks like there are issues with the design in general.
(as for the keys management - there are many solutions allowing reading public keys from ldap - that solves some of the problems)
The restriction with Host is that otherwise an untrusted host (or someone having compromised your remote account) could use your agent forwarding socket to identify as you.
That's in the user's home directory. Which is stored on the network ...
The other antipattern is defining a passphraseless key for root@serverA and putting that in authorized_keys on serverB. This is trememdously useful but insecure.
> How do you manage ssh keys once deployed? From personal experience using both systems, my thinking is that once you start deploying ssh keys, in a larger user/server environment, you have a massive rats nest. There is little to no life cycle management.
Same way you manage any other keys - configuration management software. "Keys are hard to manage" is not a valid argument against "you should not be using passwords or fail2ban".
Maybe I'm missing something, but where does fail2ban come in on the "keys vs. passwords" debate? Do I become more vulnerable if I run fail2ban on a keys-only ssh setup? If not, then why is the argument "don't use passwords or fail2ban" rather than just "fail2ban doesn't provide adequate protection for allowing passwords on ssh?"
You can do both and use denyhosts instead of fail2ban. That one doesn't even require multiple authentication failures. Known-bad hosts are blocked because enough other people blocked them: http://stats.denyhosts.net/stats.html
I agree even though I'm not sure the problem is on the "using passwords" side. There are more fundamental problems with fail2ban & Co.
Using regular expressions on logs and triggering such massive actions like IP-based bans isn't a wise thing to do. Logs are not the trivial lines of text they were 10 years before. In reality you'd have to deal with chaotic stuff often even including user-entered data - this is why there are so many classical Big-Data programs and algorithms are joining the party here. It's very hard to write solid regular expressions on those without overseeing a tiny DoS possibility because of injection. There are numerous examples in the CVE database for exactly this when you look at fail2ban and other programs using the same scheme...
If you really need to keep your logs clean and there is no way in detecting "unusual behaviour" by filtering out all those background noise you see on SSH usually, why not use iptables with it's extensions instead? You can easily count the attempts to connect to defined ports and block any IP address reaching a threshold for a limited amount of time. This is a way more solid implementation of the same thing as IP packages are well-defined structures and there is simply no way in injecting something in the IP source field of them.
About the hardware consumption by SSH login attempts: you guys aren't honestly arguing that a cronjob-like grep of log-files and performing regular expressions on hundreds of lines is a way better to do the same, right? This is actually highly inefficient too...
For me, even plain iptables filtering isn't a nice thing. I use port-knocking to get access to my SSH services. Every server has it's unique sequence of packages to tell it to open access to SSH for a IP transmitting the right packages with the right flags and payload in the correct ordering.
I'm not saying "get rid of fail2ban", but people should know what they are doing and what problems come with those programs. The convenience has it's price...
Note that Debian release do not install sudo by default, unlike Ubuntu.
So if you have not installed sudo, or do not have permission to use it, then you should use "su - " to become root, then run "apt-get update;apt-get upgrade".
58 comments
[ 2.8 ms ] story [ 261 ms ] threadThe data set contains the IP addresses for attacker and victim, the date, and the service name for each of a couple million attack reports.
edit: https://int80k.com/ftb/
Plotting your data on a world map would be kinda neat. D3.js is a good tool for that. Animate it over time for even more bonus points. :-)
Or, scrub your information from the data and post it for others to mess with.
I have several thousand entries for ssh, ssh-root, and spam abuse in my badhosts table.
You wouldn't have to query every IP, you'd just have to query IPs not covered by a result range from a previous lookup.
I'd be willing to help on this; I have a whois module I wrote for another spam tool, I should be able to pretty easily adapt it for this.
I did a first take using Kartograph: https://int80k.com/ftb/
And I see now why you're not wanting to release the data: I thought this was data on your network. Good job grabbing that email address.
Pretty funny, and clever too.
«DShield is a community-based collaborative firewall log correlation system. It receives logs from volunteers world wide and uses them to analyze attack trends.»
check the Country-Page
Waiting for multiple authentication failures to mark a host/net as "bad" is fundamentally a bad idea. Stop using passwords.
It's also useful for applications where you have to use passwords or where you need to allow anonymous requests but you don't want attackers using up all of your resources. VoIP (public-facing and handset-facing SIP) servers are my example of both of those situations.
If someone's out to get you, then sure, they'll find the port.
If someone's out to just find any unlocked door, they'll just move on.
In my case, I also use it to prevent other abuse: bans for ssh abuse protect other hosted services, and fail2ban has been modified to act on other forms of abuse too (like spam).
It is amazing how much this needs to be repeated, but security is not an absolute. Fail2ban adds another layer of security, thus it is a good thing.
tools like fail2ban are not only for SSH!
I also use log monitoring and banning to ban http connections to my web server (driven by scanning apache logs), as well as SMTP connections (scanning Exim's rejectlog).
The rationale "do not use password authentication" does not apply to unauthenticated HTTP and SMTP requests.
Fail2ban works and should be deployed regardless of whether you are using usernames/passwords or keys. I just checked one of our bastion hosts, and there are 384 banned IPs (on a rolling 30 day basis). This info is useful, and can be fed into other systems, like edge network devices.
There is actually a good article about this @ http://www.networkworld.com/article/2164048/tech-primers/ssh...
> mesh of key-based access is so dense that it is highly likely that an attack can spread to nearly all servers in an organization
Why would you ever do that? Keep the keys where the users are. There is no mesh unless you distribute the private keys and you should never do that in the first place. ssh-agent can be used if you really need to have multiple hops.
> odds are the virus will infect nearly all servers [...] including disaster recovery and backup systems that are typically also managed using such keys
Why would you store private keys to your DR/backup servers all over the place? Backup is safer if it works via pull.
There are problems with SSH key management, but the situation they describe looks like there are issues with the design in general.
(as for the keys management - there are many solutions allowing reading public keys from ldap - that solves some of the problems)
Possibly you mean, in one's ~/.ssh/config:
The restriction with Host is that otherwise an untrusted host (or someone having compromised your remote account) could use your agent forwarding socket to identify as you.That's in the user's home directory. Which is stored on the network ...
The other antipattern is defining a passphraseless key for root@serverA and putting that in authorized_keys on serverB. This is trememdously useful but insecure.
Same way you manage any other keys - configuration management software. "Keys are hard to manage" is not a valid argument against "you should not be using passwords or fail2ban".
You can have batch processes push them out periodically.
You can put public keys in LDAP
You can configure SSH to use Kerberos.
Using regular expressions on logs and triggering such massive actions like IP-based bans isn't a wise thing to do. Logs are not the trivial lines of text they were 10 years before. In reality you'd have to deal with chaotic stuff often even including user-entered data - this is why there are so many classical Big-Data programs and algorithms are joining the party here. It's very hard to write solid regular expressions on those without overseeing a tiny DoS possibility because of injection. There are numerous examples in the CVE database for exactly this when you look at fail2ban and other programs using the same scheme...
If you really need to keep your logs clean and there is no way in detecting "unusual behaviour" by filtering out all those background noise you see on SSH usually, why not use iptables with it's extensions instead? You can easily count the attempts to connect to defined ports and block any IP address reaching a threshold for a limited amount of time. This is a way more solid implementation of the same thing as IP packages are well-defined structures and there is simply no way in injecting something in the IP source field of them.
About the hardware consumption by SSH login attempts: you guys aren't honestly arguing that a cronjob-like grep of log-files and performing regular expressions on hundreds of lines is a way better to do the same, right? This is actually highly inefficient too...
For me, even plain iptables filtering isn't a nice thing. I use port-knocking to get access to my SSH services. Every server has it's unique sequence of packages to tell it to open access to SSH for a IP transmitting the right packages with the right flags and payload in the correct ordering.
I'm not saying "get rid of fail2ban", but people should know what they are doing and what problems come with those programs. The convenience has it's price...
http://dcid.me/texts/attacking-log-analysis-tools.html
It had a similar vuln many years ago.
sudo apt-get update
sudo apt-get install --only-upgrade fail2ban
Although it's considered best practice to upgrade all packages, like this:
sudo apt-get update
sudo apt-get upgrade
So if you have not installed sudo, or do not have permission to use it, then you should use "su - " to become root, then run "apt-get update;apt-get upgrade".