Just let that number sink in a bit. 80M/2844 = ~30K accounts / breach.
And still, just about every service that you have to make use of (your HOA, your insurance company, your government and so on) wants you to trust them with your data secured by a userid and a password that they will administer on infrastructure that they are most likely not competent to secure.
One of the best things to come out of the GDPR is the reporting requirements for breaches. You can expect a lot more of these updates in the future.
It's amazing how little the breaches matter, too. We like to gnash our teeth whenever a breach happens, yet somehow the breaches matter less than, say, getting into a fender bender.
To a point. Imagine if some crafty hacker leaks Slack's entire message history across all companies. Think of the chaos.
Kind of weird to consider that maybe we haven't seen a catastrophic breach yet.
Just to add on top of that the OPM breach was and still is pretty serious. A lot of personal information for a lot of people was leaked and is easily in the hands of malicious state and individual actors. The sheer scale of the breach and the amount of information out there can be leveraged by a lot of people.
The impact is often not seen immediately as databases of leaked accounts can be used for further hacks (impersonation, social engineering, blackmailing etc.) I'd attribute that to the lack of studies and difficulty to trace hacks back to leaks.
The link on Troy‘s Homepage gives me the fail page also. Seems to be a problem on the Blog page and not on HN. At least I tried the link when I posted to HN, and it worked then.
I wrote
Troy a message, but I think it’s around midnight in Queensland and he might be offline.
If you look at the list of the 3000 breaches in the blog post, these are mostly small websites. I am sure the majority of those are still unaware they are breached.
I remember a few years ago creating an account on a small online clothes shop with a unique randomly generated email. I almost immediately started receiving unrelated spam on that email. Clearly their database had been breached. I still haven't received any notification on haveibeenpwned for that email. I am sure there is an enormous mass of breaches of small mom and pops sites that are out there, unknown to both the users and the owner of the sites.
Yes, I would not be surprised at all if the majority of breaches went undetected, which means the number of them that are unreported will be very large as well.
I really do not like making accounts, especially where they are not required, such as when ordering something from some shop that I only intend to use once, or for my mobile phone company, who in their infinite wisdom have decided that having my email address is not enough to send me invoices, they need me to receive an alert that they have posted an invoice in their internal system for which they helpfully created an account that they mailed me the password for.
It really makes me wonder who comes up with these ideas.
John here from ghost.org/fail - not sure what happened there! Can't reproduce, but sounds like it was a temporary DNS resolution glitch. Will keep an eye on it
Yeah, sounds a lot like a CF DNS resolution problem at one particular edge node. Are you able to stop the browser on the CF error page and copy/paste the RayID for me? Would be helpful for us to report upstream and ask CF to flush their DNS cache at that location :)
I worry we're approaching a situation where everyone just assumes their data is out there, lethargy / ignorance of the users sets in stone, and as on the whole we don't make any progress security wise.
It will change when those people suffer very real consequences of the endless attempts of data mining -- such as being denied health insurance for something you posted on facebook.
I don’t think we’re in that situation yet. I think the average person is quite surprised what can be found about them by Googling their own name (or user handle) and looking past the first pages of Google results. How many people assume that their familial relationships and all their past addresses is easily findable using one of the free people-search services?
I applaud the dedication and effort that the author puts into HIBP.
One small suggestion, though: I wouldn't harm to ask somebody to proofread public statements like this, especially since this is a highly trust-related topic. The density of typos is annoying to me even though I'm not a native speaker.
No way for me to verify it because I use different username on different sites, and I have a self-hosted catch-all email server, I would have to extract all my usernames or emails from KeePass and check each of them separately... I'm too safe to even know I was hacked.
That might be even more interesting actually. Seeing as you could identify the exact site that was breached. That sort of data would actually be quite valuable to this project I would surmise.
I also have a similar system now as you do, but really only for higher value accounts, not the niff naff forums stuff. And like you it's all randomly generated unique passwords so pretty safe.
None of my site specific emails have been breached yet, which is good as that would mean a major global provider had been breached.
> That might be even more interesting actually. Seeing as you could identify the exact site that was breached. That sort of data would actually be quite valuable to this project I would surmise.
I have a setup where I have unique emails for each site I register with, but they are all under a specific domain. I was wondering how I could make use of HIBP and this helps immensely – thank you for sharing!
7 spanning 6 years. What's interesting, is that is the account I use for the more mainstream sites. The accounts I use for sites I don't trust, all has 0.
Hey PuffinBlue, my email is in 14 breaches :D so yeah, I beat that. (my email is in my profile, you can check)
2 years ago I started using 1password and "user+sitename@domain.com" so I'm no longer worried. I assume all accounts that I haven't migrated are compromised.
I don't recognise any of the hosts in that list. It's going to keep me awake now wondering which one of them I had an account with that was leaked. Might go find a torrent and do some spelunking ...
This information isn't valuable in any way to affected users without an easy way to verify. Obscuring that information doesn't protect the users, it only possibly sends them to places on the web where they are at a higher risk for some other security breach. This is just click bait.
For example, if the names of the files included in the zip file may have been a domain name. From seeing the contents of these dumps before, files are often named in that manner. If they are, then I as the user can then determine where my account was breached.
This of course is just one example, and not guaranteed to be an accurate one in this case.
When an email comes up as compromised in this dump, it would be great if that notification came with the filename(s) in that list where the email was found.
Especially in this case I think including the filename would have been nice; manually going through list of 2.8k domains and trying to guess if you ever have happened to have an account there is no fun. Having the filename might also help verify the veracity of the information.
I suppose the reason for omitting the filename is that Troy does not want to cast any unwarranted trust in the filenames, which is fair.
My first guess was that by publishing the file-name, it is easier to link a compromised email to a password.
I'd guess that, if you had access to the dumps, this functionality would be about a 10-20x speedup to lookups of the password belonging to an email.
However, I think the real reason is privacy. If I recall correctly, there are certain breaches where HIBP redacts the website to protect privacy. A great example of this was ashleymadison. It kinda sucks when anyone that knows your email can learn you were on that site.
It seems excessively hard to screen the entire list of 2800 items for such privacy concerns.
he publishes the list of domains in this article. I've already recognized one i had an account on, and was able to notify the owner - it looks like it might have been from a known 2016 hack
Simply knowing is usually good enough for me - nearly anytime my email is identified in a potential breach, I assume it is time to check my accounts and roll my passwords.
And just because XYZ site was previously listed as a breach, doesn't mean it wasn't breached again with your new password
Yes, but you gotta find a balance - you'll go crazy if you rotate passwords everyday! Seeing your email in a new breach is certainly a good time to do so.
There should be a way to check entire domains, outside of known public email providers. I use a different email address for every website I use and a catch-all rule on my domain to redirect them all to a single email account. I won't check them all simply because I don't even remember them all. I know this could have the downside of being useful in email address enumeration though.
I'm a little surprised he doesn't share actual password lists with at least some other security researchers. I get there are legal issues involved but I thought researchers had legal protections?
"Dark web" (lol) hackers are already going to be circulating the sources of this list and in any case could probably crack sha-1 reasonably quickly. So if you do a minimal amount of verifying the person who asks (e.g. ring their university) you can surely help the good guys with little risk of misuse?
Most of them will be known already. It's mostly just a matter of comparing hashes or at worst converting plaintext SHA-1 (which is designed to be very fast).
And crackers will share unknown hashes to be cracked. So they can crowd source cracking far more effectively than the security community can.
In addition, the list isn't connected with any specific accounts* - even if you broke the whole list (which it essentially is already, anyway) someone would have to match each one to some rando account. Not likely.
*unless you have the original breach, then we are back at zero: its already compromised.
Indeed. I guess the best practical argument is sharing them securely is hard. Which is very true. However if it's not possible for two security researchers to exchange data then what hope is there for the rest of us?
I get he doesn't want to do that and that is his prerogative. However, it does feel like we're so scared of things falling into the wrong hands that we hobble our ability to defend against hacks in the first place.
I would imagine if there are any security researchers he trusts sharing the data with, they wouldn't need to email him out of the blue for it, nor need to discuss it with the rest of us.
I received a notification earlier, though only regarding a dead email address that I forward, it's probably some forum I signed up to ages ago and forgot about.
I guess this is also a good time to remind the general population that password managers are a thing and prevent such breaches from turning a "oh I don't visit that website anymore" into "oh sh*t my bank account just went to 0"
It's getting to the point where it feels like the site could just be replaced by one of those joke sites where the answer is just a static string: Yes.
Hopefully at least it will push more people towards password managers and avoiding password reuse.
> I think that would be a much more interesting and useful way to spend two hours, rather than implementing cute little algorithms inside an isolated environment like HackerRank.
Hiring manager here, one who doesn't use HackerRank either. I'm in agreement with the premise, but I would likely decline an applicant unironically employing the tone of writing emphasized above given the implications against cooperation and teamwork.
"cute little" can be replaced with "case-specific" or "niche," maintaining the same general meaning while divorcing it from adversarial connotations.
So my most-breached e-mail address (a hotmail address from 17 years ago that I basically use for stuff I know will send me spam) has 3 or 4 newly-added breaches.
2,844 Separate Data Breaches (unverified): In February
2018, a massive collection of almost 3,000 alleged data
breaches was found online. Whilst some of the data had
previously been seen in Have I Been Pwned, 2,844 of the
files consisting of more than 80 million unique email
addresses had not previously been seen. Each file
contained both an email address and plain text password
and were consequently loaded as a single "unverified"
data breach.
Compromised data: Email addresses, Passwords
The problem is I have no idea what this breach is sourced from, so I don't know what account in particular was affected. Not Troy's fault, obviously, but it's basically just telling me, "Your e-mail address and a password to somewhere was found online somewhere." For all I know it's recycled data from a breach I already know about.
At this point it's not really a big deal. I use a password manager and they're all random at this point, so I just need to figure out which one to rotate.
Same. I got the email this morning and looked to see what I need to do. Turns out, what I need to do is just, like, be wary? No way to know what accounts I ought to go change passwords for. Seems unreasonable to go change literally all of them over this, that would take forever. This news makes me panicky but gives me no reasonable course of action.
I wrote a little python script to match the websites with the Alexa Top 1 Million and sort by rank. I found 268 top 1 million sites in the pwned list. You can check out the ranking here if you want https://github.com/tail-recursion/hibp-rankings/. Notable websites include; lotro.com, mtgox.com, malwarebytes.org, daemon-tools.cc, autohotkey.com, vbulletin.org
One of the nice things about changing email addresses... every single notification I've ever gotten from this site applies to my old addresses. A couple years after switching, no breaches of my current credentials have appeared yet.
Eh, shouldnt a email-service, actually have some honeypot-accounts. Accounts starting numbers or a A at the start of the name. So when another service is breached, any user-account reciving email from the compromised service, gets quarantined instantanously- and the user gets send a one-time key to his cellphone, to change his password?
I ran the emails of a few of my family through the pwned API so that I could tell them to change their passwords. I was surprised that almost everyone I reached out to was surprised by their information being out there. My interpretation was the general public is almost entirely unaware of the scope of these breaches and what they need to do as a result.
It'd be cool if there was some way of gently sharing with your friends and family whether or not they've been pwned -- maybe some kind of social network plugin or web app that generates emails for you to send to them. A key challenge would be being sensitive to their privacy, i.e., not coming off as creepy.
99 comments
[ 2.7 ms ] story [ 165 ms ] threadAnd still, just about every service that you have to make use of (your HOA, your insurance company, your government and so on) wants you to trust them with your data secured by a userid and a password that they will administer on infrastructure that they are most likely not competent to secure.
One of the best things to come out of the GDPR is the reporting requirements for breaches. You can expect a lot more of these updates in the future.
Edit: interesting, the link now goes to 'https://ghost.org/fail/', a 404, so go to https://www.troyhunt.com/
To a point. Imagine if some crafty hacker leaks Slack's entire message history across all companies. Think of the chaos.
Kind of weird to consider that maybe we haven't seen a catastrophic breach yet.
People died because of that one:
https://en.wikipedia.org/wiki/Ashley_Madison_data_breach#Imp...
[0] https://en.wikipedia.org/wiki/Office_of_Personnel_Management...
[1] https://www.wired.com/2016/10/inside-cyberattack-shocked-us-...
[2] http://www.slate.com/articles/technology/future_tense/2015/0...
I wrote Troy a message, but I think it’s around midnight in Queensland and he might be offline.
https://webcache.googleusercontent.com/search?q=cache:jLWeTh...
I remember a few years ago creating an account on a small online clothes shop with a unique randomly generated email. I almost immediately started receiving unrelated spam on that email. Clearly their database had been breached. I still haven't received any notification on haveibeenpwned for that email. I am sure there is an enormous mass of breaches of small mom and pops sites that are out there, unknown to both the users and the owner of the sites.
I really do not like making accounts, especially where they are not required, such as when ordering something from some shop that I only intend to use once, or for my mobile phone company, who in their infinite wisdom have decided that having my email address is not enough to send me invoices, they need me to receive an alert that they have posted an invoice in their internal system for which they helpfully created an account that they mailed me the password for.
It really makes me wonder who comes up with these ideas.
Also, I just re-tested and it is still not working.
Your IP address: 1.2.3.4
Error reference number: 524
Cloudflare Location: Seattle
Another possible hint: opening the link in a FF private window works.
One small suggestion, though: I wouldn't harm to ask somebody to proofread public statements like this, especially since this is a highly trust-related topic. The density of typos is annoying to me even though I'm not a native speaker.
My host 'hacked' email address has been involved in 6 breaches.
Anyone beat that?
I also have a similar system now as you do, but really only for higher value accounts, not the niff naff forums stuff. And like you it's all randomly generated unique passwords so pretty safe.
None of my site specific emails have been breached yet, which is good as that would mean a major global provider had been breached.
That's why I do it.
https://haveibeenpwned.com/DomainSearch
Probably someone just making up an address randomly but three times is a bit odd.
If some small site gets hacked there is a much smaller chance that it finds its way to Troy.
Sure: Zero ;)
But that's no real surprise, I try to keep the number of accounts I have to an absolute minimum which is one of the easier ways to avoid this fate.
2 years ago I started using 1password and "user+sitename@domain.com" so I'm no longer worried. I assume all accounts that I haven't migrated are compromised.
This of course is just one example, and not guaranteed to be an accurate one in this case.
I suppose the reason for omitting the filename is that Troy does not want to cast any unwarranted trust in the filenames, which is fair.
However, I think the real reason is privacy. If I recall correctly, there are certain breaches where HIBP redacts the website to protect privacy. A great example of this was ashleymadison. It kinda sucks when anyone that knows your email can learn you were on that site. It seems excessively hard to screen the entire list of 2800 items for such privacy concerns.
you can also search now for your addresses and see if you were affected.
And just because XYZ site was previously listed as a breach, doesn't mean it wasn't breached again with your new password
"Dark web" (lol) hackers are already going to be circulating the sources of this list and in any case could probably crack sha-1 reasonably quickly. So if you do a minimal amount of verifying the person who asks (e.g. ring their university) you can surely help the good guys with little risk of misuse?
> in any case could probably crack sha-1 reasonably quickly.
So that shouldn't bother you then.
And crackers will share unknown hashes to be cracked. So they can crowd source cracking far more effectively than the security community can.
That suggestion likely reflects reality.
*unless you have the original breach, then we are back at zero: its already compromised.
1Password then did a POC integration into their service so you can test if your password shows up in HIBP data.
https://blog.agilebits.com/2018/02/22/finding-pwned-password...
https://www.troyhunt.com/no-i-cannot-share-data-breaches-wit...
https://www.troyhunt.com/here-are-all-the-reasons-i-dont-mak...
I get he doesn't want to do that and that is his prerogative. However, it does feel like we're so scared of things falling into the wrong hands that we hobble our ability to defend against hacks in the first place.
But no, there is likely no hope, lol
I guess this is also a good time to remind the general population that password managers are a thing and prevent such breaches from turning a "oh I don't visit that website anymore" into "oh sh*t my bank account just went to 0"
Hopefully at least it will push more people towards password managers and avoiding password reuse.
line 713
Hiring manager here, one who doesn't use HackerRank either. I'm in agreement with the premise, but I would likely decline an applicant unironically employing the tone of writing emphasized above given the implications against cooperation and teamwork.
"cute little" can be replaced with "case-specific" or "niche," maintaining the same general meaning while divorcing it from adversarial connotations.
At this point it's not really a big deal. I use a password manager and they're all random at this point, so I just need to figure out which one to rotate.
https://inspiredelearning.com/resource/create-strong-passwor...
It'd be cool if there was some way of gently sharing with your friends and family whether or not they've been pwned -- maybe some kind of social network plugin or web app that generates emails for you to send to them. A key challenge would be being sensitive to their privacy, i.e., not coming off as creepy.