Hmm. Given the amount of information removed (justifiably so), there isn't much left to analyze other than aggregates. (Which the dashboard mockup in the post already does)
What might be interesting is graphing a network of the relationship between services which suffered a breach. Fortunately, I have the proper workflow prepared to process the data in this manner, so I'll take a look.
So it would appear the most interesting data point on these records is the date and time of the breach discovery, or using that to estimate a breach date? Correlating that with some other publicly available data sets (e.g.: historical routing data) might yield some useful results?
Yeah. I would've rather had a dump of just the passwords -- no usernames, site names, service names, counts, deduplication, just 1.4 billion passwords, one password to a line.
I'd feed it into my bad-password-indexing program (https://github.com/robsheldon/bad-passwords-index) and it would spit out a file that could be used to improve password security in a not-entirely-stupid way somewhere.
There are enough plain-text password lists for that, rockyou (old but good) and there are many more. Kali distro has some of them shipped if you are interested, and many more can be found online.
That's a tool they already have plenty of access to. I've tried to make a tool that will work against them, making it possible for any service, running any software stack, to efficiently blacklist hundreds of thousands of common passwords.
I can (and should) start tracking down more password dumps and compiling them myself to do the same thing, but I've been spread pretty thin for a while. It's been on my mind but there hasn't been time for it.
A massive list of plaintext passwords from Troy's dump, the ones that were available anyway, would have been really convenient. Odds are my index file would already be updated to a newer version.
I thought all his data came from publicly available dumps. If so, it might make it easier to for them to have it all in one place, but it's not like he's giving them anything they can't get if they really wanted it (and probably have already).
No, it also contains non-public dumps that someone gave to him (after authentication) iirc. But he never pays for dumps. I think he posted an explanation on his blog a while ago.
The difference between your first draft[1] and mine is that I've only included links where each site had at least 1% of users in the other site. This helps make links more meaningful.
For instance, about 37k users were members of both Tumblr and Plex. Tumblr had the largest number of Plex users, so it should be a strong link, right? No: because Tumblr has such a large userbase, this is a relatively insignificant relationship; 0.14% of Tumblr users had Plex accounts, which ranks only 20th for Plex.
Compare this to Plex and Xbox-Scene. 5.6k users had Plex and Xbox-Scene accounts, which is about 2% of each site's userbase. I believe this link is much more meaningful than the Plex-Tumblr link, and I believe my visualisation bears this out.
I'm always amazed at how efficient AWS is at detecting private keys on the internet (checked into github, etc..), and then proactively locking down accounts. I wonder how long it will be until we see a similar service from consumer accounts, like github or twitter. Seems like "have I been pwned" might offer a commercial API for such benefit...
I used to be somewhat interested by stats on passwords, etc. from breach data dumps.
haveibeenpwned is a helpful and legit site, though I think it should have used email confirmation instead of requiring only an email address.
I also respect Troy along with many other security researchers. Even those that are up to no good in the security world in some ways have contributed good things; after all, the rest of us are stronger and more vigilant now than we used to be because of their work.
However, this anonymized data will almost certainly be used by black hats more than white hats, and I don't see how this release is good for the majority of those that were affected by these breaches.
Which in and of itself is silly. The raw dumps are already available to everyone, blackhats included. Personally I'm tempted to make a site that just lists it by domain and stuff. I found several people at my company with Ashley Madison accounts using a quick grep.
Casually looking up your peers is exactly what you should be doing in my opinion. But I'm not a good person and I'd rather see details like that plastered everywhere. Be an idiot, get what you deserve.
I have a 2004 Gmail account which people like to use as a fake email address when signing up to things. So my address is in the Ashley Madison leak. This is not something I'm comfortable with people knowing without context, so even if it was a policy of "play stupid games, win stupid prizes" I'd get harmed.
I'm with you that haveibeenpwned should require email confirmation before listing the breaches an email address was in. As it is, it has become an easy to use dictionary of places to find personal data for any given email address. Yes it doesn't list the breaches that Troy has deemed "sensitive", but if you're trying to commit identity theft or fraud using personal data, it doesn't really matter if these search results don't include websites that deal with porn/adultery/children.
Doubtless there are black market tools that provide such a service but expose far more data, but haveibeenpwned lowers the barrier to entry significantly by being far more available to the public.
I have wanted to ask this for a long time. What do / should you do once you have been pwned?
You definitely have to change password, or even use password manager. But your record is now widely available it feels as you have been naked on the Internet.
You may likely change your email address ( Login Name ). But opening another email account is such an hassle. An opening yet another account on those of your favorite site means you lost all of your pass record.
What are you worried about here? Further breaches? If you've changed your password, there's not much more you can do. Getting a new email address doesn't make you more secure. If you're trying to conceal the fact that you have an account somewhere, you should not be using your email address there in the first place.
Saw this on HN earlier in the year and have been slowly phasing into my life. I have mail setup for my domain and assign per-site aliases (hn@mail.com, fb@mail.com, etc). All of the mail just forwards to my gmail that I've had forever, though that is more of a matter of convenience and habit. The key thing is, I can spin up a new email whenever I want since I am the wildcard, and I control the host and am not tied to the whims of gmail so I am covered on both ends.
I still want the dumps for myself, so I can determine if any of my accounts are compromised. I understand why it's not possible to distribute these dumps, but for the same reason, I'm not going to search someone else's database. How can we have a database of dumps that people can query safely, without revealing their information to the people with the dumps?
38 comments
[ 3.5 ms ] story [ 93.2 ms ] threadWhat might be interesting is graphing a network of the relationship between services which suffered a breach. Fortunately, I have the proper workflow prepared to process the data in this manner, so I'll take a look.
I'd feed it into my bad-password-indexing program (https://github.com/robsheldon/bad-passwords-index) and it would spit out a file that could be used to improve password security in a not-entirely-stupid way somewhere.
I can (and should) start tracking down more password dumps and compiling them myself to do the same thing, but I've been spread pretty thin for a while. It's been on my mind but there hasn't been time for it.
A massive list of plaintext passwords from Troy's dump, the ones that were available anyway, would have been really convenient. Odds are my index file would already be updated to a newer version.
I might be onto something. It appears the communities algorithm correctly detected the Russian and China communities.
The difference between your first draft[1] and mine is that I've only included links where each site had at least 1% of users in the other site. This helps make links more meaningful.
For instance, about 37k users were members of both Tumblr and Plex. Tumblr had the largest number of Plex users, so it should be a strong link, right? No: because Tumblr has such a large userbase, this is a relatively insignificant relationship; 0.14% of Tumblr users had Plex accounts, which ranks only 20th for Plex.
Compare this to Plex and Xbox-Scene. 5.6k users had Plex and Xbox-Scene accounts, which is about 2% of each site's userbase. I believe this link is much more meaningful than the Plex-Tumblr link, and I believe my visualisation bears this out.
[1]http://i.imgur.com/FpLiFGk.png
* Note: since I ignored all lines with only one site, this "1%" only includes users who were in more than one breach!
haveibeenpwned is a helpful and legit site, though I think it should have used email confirmation instead of requiring only an email address.
I also respect Troy along with many other security researchers. Even those that are up to no good in the security world in some ways have contributed good things; after all, the rest of us are stronger and more vigilant now than we used to be because of their work.
However, this anonymized data will almost certainly be used by black hats more than white hats, and I don't see how this release is good for the majority of those that were affected by these breaches.
Doubtless there are black market tools that provide such a service but expose far more data, but haveibeenpwned lowers the barrier to entry significantly by being far more available to the public.
You definitely have to change password, or even use password manager. But your record is now widely available it feels as you have been naked on the Internet.
You may likely change your email address ( Login Name ). But opening another email account is such an hassle. An opening yet another account on those of your favorite site means you lost all of your pass record.
I would love to know which DNS provider has the best security if anyone has done the research.
Running your own registrar gets tricky, however.
I've come to assume some non-zero percentage of sales/marketing/lead gen companies are using this as a go-between for an SMTP validator.
Or even better, build the service you're asking for! Clearly, there's a need.