Because dutch news outlets always contact them for quotes, because they only know one security company. I'm pretty sure they are not the best.
I tend to find it disappointing to see a security company use google analytics on their site and leak their visitor information to a big data third party.
That is a very hard question. The simple answer is _that one_ security company that manages to compromise you and point out all issues within your company and or code for you to fix/address, that's the best one (for you). (if we can leave the financial part out of it, because security advice or audits aren't cheap either.)
But it is not that simple. I don't consider that very likely, that you actually find that 'right' company. (and putting faith in a third party to audit your code/company, brings other risks too). And chances are that some issues will be overlooked but not by that 'weird' guy/girl in the attic that has all the time of the world to security probe you 'for free'.
And there will always be a part you can't audit properly, your employees, which often is the origin of a hack, disgruntled employees. Or even worse, a disgruntled employee of the security company that just audited you.
And keep in mind, with enough power, you can get anything offline or in a non acceptable state (half working, data compromise etc). So security companies can only help you a certain length, nonetheless it can be very helpful to have a person from outside look at things with 'fresh eyes' and see things you have overlooked for so long because you are right in the middle of it all.
I have no idea if they're the best, but they're pretty important. They also handle lots of internet security for the government. Them getting in trouble is a big deal.
The term security expert (or similar) always gives me this uncomfortable feeling. Even if they don't literally call themselves this, definitely (in the case of Fox-IT) their appearance in the media often portrays them like this to the general public.
Without getting into semantics, the general public is often left with experts' advice which makes them feel incompetent and/or scared. I guess you could blame journalism, but I don't know if you can expect them to be the whole bridge between both (that's another philosophical discussion anyway).
My beef with these 'security experts from Fox-IT say...'-pieces is they don't provide proper context. Something like technology is complex, security is complex, it is hard or impossible to know everything, to know future risks,... you get the picture. I really like it when people acknowledge they don't know everything and in this case specifically because the opposite, I think, has a very negative effect on the general public.
As Fox aka (Twenty-First Century Fox, Inc.) got acquired by Disney yesterday and is in the HN news. This news (Fox-IT) is about a completely different company from another continent!
"Disney to Acquire Twenty-First Century Fox" (thewaltdisneycompany.com)
533 points by mxfh 23 hours ago
Fox-IT is unrelated to Fox, so it's not the internal IT department of Fox. Fox-IT seems to be a company from Netherland.
@downvoters: what's your problem? I wasted 5min reading the news, until I found out that it's a minor known company, and not related to "Fox". And then I inform others on HN, but then I get downvotes, yeah right - toxic
> The name of the country is the Netherlands, not Netherland.
I disagree. I'm Dutch[0], and I find it stupid that English speakers always pluralise our country. We've been a unified country for quite some time now. Netherland is fine.
[0] Yeah, "Dutch" is another one of those weird things that English inflicts on us. Why?
But "Netherland" is not an English word, and is likely to lead to a double-take in conversation, if not outright confusion.
English, like all languages, retains a lot of legacy for a very good reason. Some changes are inevitable, but maintaining continuity is very useful in keeping the language readable in the future.
As for the label "dutch", what do you propose for a substitute?
It was a republic of somewhat independent provinces once, but not anymore. It's a single country. In Dutch nobody calls it "De Nederlanden". We live in Nederland. The only reason other languages refer to it in plural is because those languages are still stuck in the 18th century in how they refer to us.
The core of the problem, a lot of paragraphs down:
> we have strong evidence that supports our hypothesis that the adversary gained access to our [DNS provider's] credentials through the compromise of a third party provider
> A factor which possibly helped the attacker was that the password had not been changed since 2013
> We chose our DNS provider 18 years ago when 2FA was neither a consideration nor a possibility
First time I used a 2FA device was an RSA "Safeword Card" which looked like a small handheld calculator and would display a multi-digit code that changed every minute or so. This was in 1998 and it had been put into use before that.
I think Sun Micro had deployed those to all their engs, perh others too. I think it had a PIN which when entered displayed the code which was valid for a min.
I meant SecurID were fairly well established by the early 2000s for VPN access, from what I remember, so I assume they came around at least a few years earlier, which would be the late 90s, they also weren't the only option on the market.
> but 2FA was both possible and a consideration 18 years ago
You could setup your own dns servers obviously 18 years ago and in fact when starting out in the mid 90's that is exactly what we and many others did. (Criket Liu nutshell books from O'Reilly)
To clarify, the problem here wasn't with their DNS servers, but rather the registrar being compromised. You could run your own DNS servers and still get compromised like this unless you also happen to run your own registrar.
To me the biggest question is whether or not TLS was in place. That should prevent the attacker from intercepting the actual data without the victim noticing.
> Maximum 10-minute time window during which the attacker temporarily rerouted and intercepted Fox-IT email for the specific purpose of proving that they owned our domain in the process of fraudulently registering an SSL certificate for our ClientPortal.
Specifically Comodo reports that they sent their normal validation email to hostmaster@fox-it.com (which unknown to Comodo or Fox-IT at the time was being directed to the attackers). I've never used Comodo's implementation of 3.2.2.4.4 but typically there's an email with a code in it, telling you to go to a web page and paste the code in if you want to authorise the issuance of the requested certificate, or something along those lines.
The security of this validation method (3.2.2.4.4) depends upon
1. You control DNS for your domain including the MX records used to deliver email (this is where Fox-It came undone here)
2. You control the MX servers, or if you have a third party providing backup MX, you trust them not to abuse that
3. The Certificate Authority does a good job of getting accurate DNS records and connecting to the right IP address
4. All email addresses in your WHOIS records plus a handful of famous ones like hostmaster@ postmaster@ are delivered to people you trust in your organisation.
No, it doesn't. As mentioned in the article, the attacker successfully requested a TLS certificate for the hostname, which was possible because he could pass a CA's domain validation.
I'm not entirely sure, but I think HPKP could have prevented this for returning customers, because Fox-IT would have been able to pin the key of their own certificate. Then the new certificate used by the attacker would have been rejected by the customer's browser.
Yes, it more highlights that the simple presence of a valid certificate does NOT guarantee that you are connecting to the service that you think you are.
EV certs would be harder to compromise, but likely not too difficult for a sophisticated attacker. And who really notices if a site that had an EV cert suddenly doesn't? I might for my bank, but likely would not for a software product website.
This is the exact reason I moved all my DNS to a new provider, as the old one does not allow 2FA. It sucks that you have to rely on an external supplier for such a critical part of your infrastructure. In my experience, most DNS suppliers mostly care about marketing, not so much about security.
I disagree with the notion that they're ambiguous at all. Foxit and Fox-IT have quite a noticeable difference in their names, IMO. (although they do have the same letters, so I get your point)
Would a screen reader pronounce it as “fox dash it” or “fox it” for both? In my mind I read “fox it” without the dash and thought it was the PDF company.
Which was really weird trying to wrap my head around how they could discover an issue like this so quickly.
>Sept 19 2017, 02:21 The actual MitM against our ClientPortal starts. At this point, the fraudulent SSL certificate for ClientPortal was in place and the IP DNS record for clientportal.fox-it.com was changed to point to a VPS provider abroad.
>Sept 19 2017, 07:25 We determined that our name servers for the fox-it.com domain had been redirected and that this change was not authorized. We changed the DNS settings back to our own name servers and changed the password to the account at our domain registrar. This change will have taken time to have full effect, due to caching and the distributed nature of the domain name system.
5 hour, 4 minute response time. I would be okay with that _if_ I could get over what was overlooked.
Another way could be to monitor the number of unique IP addresses connecting? That will get harder with IPV6, where you'll probably want to do it on subnet level.
Do you have statistics about how fast different CAs are/how it is distributed? Are some CAs always "lagging", or do many of them have occasional late outliers, ...?
That would be a great thing to analyse, but I don't currently have stats.
Note, Google crawler will also submit anything it sees, so if your CA lags by a week but your new site is crawled in the next hour, you'll end up logged.
Note that this doesn't tell you whether a CA logs all/ most or none of the certificates they issue. In my experience it would be unusual for a CA to deliberately _wait_ before logging certificates, either they're logged more or less immediately or they don't intend to log them at all. To issue certificates with an SCT baked inside them (which is convenient for customers) you have to log a "pre-certificate" with the exact same details first anyway to get the SCT.
BUT if there's a problem the Mozilla trust store policy (and perhaps others, but only Mozilla's happens in public where we can see it) says the CA must show the trust store all the affected certificates. By far the easiest way to do that in 2017 is shove them all into a CT log and then spew a list of links to crt.sh or another monitor, rather than uploading some Word document full of X.509 PEM files or whatever. So even CAs that we know don't normally log everything will put all the problematic stuff into logs during disclosure, or else someone reading m.d.s.policy will do it for them.
Some CAs have a deliberate customer policy of letting you choose NOT to log, sometimes with a warning saying if you don't log this stuff then Google might distrust it. Symantec was really into redaction, which is logging but with the "sensitive" bits of the certificate removed. But Google never really bought that idea, and Symantec are exiting the CA business.
Once a certificate is submitted to the logs (which as mentioned elsewhere, Google wants to make mandatory, but today they only enforce for EV and only by treating as non-EV if it isn't logged) the log won't necessarily immediately tell monitors about that certificate. Accepting new items for the logs can be (and usually would be) parallelised, but the log structure is a single Merkle Tree and so cannot be calculated entirely in parallel, usually updating this structure is a serial operation that happens asynchronously, with the result available some time later.
A policy value called the Maximum Merge Delay decides how long a log has to get this done and produce a new head value for the log, Google has currently chosen 24 hours for logs to be accepted in Chrome. Once a log presents somebody with an SCT proving it logged a particular certificate, it has 24 hours to make a Merkle Tree with that cert in it available to the monitors. That sounds like a long time, but it's not "business hours" or "best effort" it's an absolute limit. Data Centre flooded by a tsunami? Too bad. OS upgrade is incompatible with your CPU model? Don't care. Some lunatic threw a billion dollars in bills out of an aeroplane and your employees are out collecting as much as they can carry? Not our problem. Usually you can expect the actual turnaround to be much less than 24 hours, if it's ever greater the log should be distrusted and shut down.
There have already been logs that had "problems" and went away for this reason. Because the logs are only useful if they obey the MMD Google is being pretty strict about this.
However, just because something is in one or more logs doesn't mean any monitor, including famous ones like crt.sh, or Google's own transparency site, has actually obtained and processed the log item right away. There can definitely be slowdowns in this part, but they're only a matter of throwing enough resources at the problem.
The affected certificate as confirmed by Comodo: https://crt.sh/?q=278968925. Note that it wasn't logged until about 2 hours ago. CT logging will become mandatory at some point, but not yet.
Weird. While they're on Cloudflare Registrar now, it appears they used to be on Network Solutions.
NetSol is pretty bad in a lot of ways, but you can make it less bad with some configuration competence. I would have assumed an IT security company would have done that.
The first six Google hits for this phrase, relate to Fox-IT being hit. Can anyone share some details on these sensors, given they are described as being critical to the response?
Some of the results I find for CTMP are Continuous Time Markov Processes. Here is a slide deck with poor kerning[0]. From an interesting slide:
Numerical solution of implicitly encoded CTMCs
If R is stored as a sparse matrix of columns, a Jacobi or GaussSeidel iteration has the cost of a vector-matrix multiplication, O(n(R))
If R is stored using implicit techniques (Kronecker algebra or decision diagrams), memory is greatly reduced but runtime can increase.
E.g., using Shuffle to compute the effect of v, (pi)^(old) . (operator)(L<k<1) R(k,v1), is slower than Sparse multiplication if the R(k,v) matrices are very sparse.
This research paper describes tracking cattle with a wireless sensor network[1].
---
Or this could be Cisco Technology Migration Program[2]. They could have some sort of network probe to identify "upgrade-ready" products, which could be leveraged for other purposes.
I'm going from (a not very good) memory but I wanna say it's something like "<something> traffic monitoring probe".
Basically, they do full packet captures of all network traffic and retain it for a certain period of time so that it can be reviewed later (such as when something like this happens).
I work for Fox-IT (but don't speak for Fox-IT here).
CTMp network sensors are servers storing and filtering network traffic, and generating alerts. We use these sensors for our own monitoring services, and provide these sensors to others as part of the Cyber Threat Management platform (software to run your own Security Operations Center, with greater or lesser involvement of Fox-IT experts).
DNS is REALLY a weak link. Another one is the border gateway protocol. I recently read about all GOOGLE, FB, Apple,etc traffic being redirected through a single Russian address for a few minutes.
So how does a third party get access to the creds to allow messing with your DNS hosting account? Social engineering? Or are there holes we need to know about?
For all forms of certified ethical hacking and formidable penetration into social media accounts, mobile phone cloning and spyware, espionage services, web server and database penetration or take down, university database upgrade or modification - geniushack08(protonmail)com
81 comments
[ 3.0 ms ] story [ 171 ms ] threadI tend to find it disappointing to see a security company use google analytics on their site and leak their visitor information to a big data third party.
But it is not that simple. I don't consider that very likely, that you actually find that 'right' company. (and putting faith in a third party to audit your code/company, brings other risks too). And chances are that some issues will be overlooked but not by that 'weird' guy/girl in the attic that has all the time of the world to security probe you 'for free'.
And there will always be a part you can't audit properly, your employees, which often is the origin of a hack, disgruntled employees. Or even worse, a disgruntled employee of the security company that just audited you.
And keep in mind, with enough power, you can get anything offline or in a non acceptable state (half working, data compromise etc). So security companies can only help you a certain length, nonetheless it can be very helpful to have a person from outside look at things with 'fresh eyes' and see things you have overlooked for so long because you are right in the middle of it all.
Without getting into semantics, the general public is often left with experts' advice which makes them feel incompetent and/or scared. I guess you could blame journalism, but I don't know if you can expect them to be the whole bridge between both (that's another philosophical discussion anyway).
My beef with these 'security experts from Fox-IT say...'-pieces is they don't provide proper context. Something like technology is complex, security is complex, it is hard or impossible to know everything, to know future risks,... you get the picture. I really like it when people acknowledge they don't know everything and in this case specifically because the opposite, I think, has a very negative effect on the general public.
"Disney to Acquire Twenty-First Century Fox" (thewaltdisneycompany.com) 533 points by mxfh 23 hours ago
https://news.ycombinator.com/item?id=15921692
Fox-IT is unrelated to Fox, so it's not the internal IT department of Fox. Fox-IT seems to be a company from Netherland.
@downvoters: what's your problem? I wasted 5min reading the news, until I found out that it's a minor known company, and not related to "Fox". And then I inform others on HN, but then I get downvotes, yeah right - toxic
- You comment this on a comment that already describes exactly what Fox-IT is.
- You imply that anything that happens on another continent than yours is time "wasted".
- Which in turn implies that all HN readers should be from the same continent that you are from.
- There are many other companies with a name that contains Fox (pdf reader, apparel), this stuff happens.
- You ask downvoters what their problem is. Which, from my experience is a very good way to get more downvotes.
- The name of the country is the Netherlands, not Netherland.
I disagree. I'm Dutch[0], and I find it stupid that English speakers always pluralise our country. We've been a unified country for quite some time now. Netherland is fine.
[0] Yeah, "Dutch" is another one of those weird things that English inflicts on us. Why?
But "Netherland" is not an English word, and is likely to lead to a double-take in conversation, if not outright confusion.
English, like all languages, retains a lot of legacy for a very good reason. Some changes are inevitable, but maintaining continuity is very useful in keeping the language readable in the future.
As for the label "dutch", what do you propose for a substitute?
It's not a word, it's the name of a country. Peking got changed to Beijing because it as more accurate, right? This is the same thing.
Look at your passport or ID card: It says "Koninkrijk Der Nederlanden" not "Koninkrijk der Nederland".
The Netherlands ( English, plural ) Niederlande ( German, plural ) Les Pays Bas ( French, plural )
The Netherlands -> "The low lands" -> Plural
edit: https://nl.wikipedia.org/wiki/Nederlanden ( plural, it was a republic once )
Singular is fine.
> we have strong evidence that supports our hypothesis that the adversary gained access to our [DNS provider's] credentials through the compromise of a third party provider
> A factor which possibly helped the attacker was that the password had not been changed since 2013
> We chose our DNS provider 18 years ago when 2FA was neither a consideration nor a possibility
Isn't the account on the card + your pin 2FA?
I meant SecurID were fairly well established by the early 2000s for VPN access, from what I remember, so I assume they came around at least a few years earlier, which would be the late 90s, they also weren't the only option on the market.
You could setup your own dns servers obviously 18 years ago and in fact when starting out in the mid 90's that is exactly what we and many others did. (Criket Liu nutshell books from O'Reilly)
http://shop.oreilly.com/product/9781565920101.do
http://www.oreilly.com/pub/au/284
[1] https://twofactorauth.org/#domains
> Maximum 10-minute time window during which the attacker temporarily rerouted and intercepted Fox-IT email for the specific purpose of proving that they owned our domain in the process of fraudulently registering an SSL certificate for our ClientPortal.
The security of this validation method (3.2.2.4.4) depends upon
1. You control DNS for your domain including the MX records used to deliver email (this is where Fox-It came undone here)
2. You control the MX servers, or if you have a third party providing backup MX, you trust them not to abuse that
3. The Certificate Authority does a good job of getting accurate DNS records and connecting to the right IP address
4. All email addresses in your WHOIS records plus a handful of famous ones like hostmaster@ postmaster@ are delivered to people you trust in your organisation.
I'm not entirely sure, but I think HPKP could have prevented this for returning customers, because Fox-IT would have been able to pin the key of their own certificate. Then the new certificate used by the attacker would have been rejected by the customer's browser.
EV certs would be harder to compromise, but likely not too difficult for a sophisticated attacker. And who really notices if a site that had an EV cert suddenly doesn't? I might for my bank, but likely would not for a software product website.
What! That's impossible! /s
An excuse right off the bat and quite poor one, especially since it is a security company.
Or are you more taking aim at it being a defeatist position? I do t fully understand.
I’ve never seen a decent, well-resourced red team not win.
Which was really weird trying to wrap my head around how they could discover an issue like this so quickly.
I mean, it clearly confused me and a lot of other people as well, so I'm confused what this means.
It's like reading Vm-Ware instead of VMWare. People would generally think they're referring to the same company unless they've heard of both.
>Sept 19 2017, 07:25 We determined that our name servers for the fox-it.com domain had been redirected and that this change was not authorized. We changed the DNS settings back to our own name servers and changed the password to the account at our domain registrar. This change will have taken time to have full effect, due to caching and the distributed nature of the domain name system.
5 hour, 4 minute response time. I would be okay with that _if_ I could get over what was overlooked.
Periodically calling the endpoint and validating the certificate used is the right one?
Also, automated DNS monitoring could work too.
Which is still highly valuable, in the scheme of things. But wouldn't improve on the timescales they've described.
(I run ctadvisor.lolware.net)
Note, Google crawler will also submit anything it sees, so if your CA lags by a week but your new site is crawled in the next hour, you'll end up logged.
https://crt.sh/monitored-logs
Note that this doesn't tell you whether a CA logs all/ most or none of the certificates they issue. In my experience it would be unusual for a CA to deliberately _wait_ before logging certificates, either they're logged more or less immediately or they don't intend to log them at all. To issue certificates with an SCT baked inside them (which is convenient for customers) you have to log a "pre-certificate" with the exact same details first anyway to get the SCT.
BUT if there's a problem the Mozilla trust store policy (and perhaps others, but only Mozilla's happens in public where we can see it) says the CA must show the trust store all the affected certificates. By far the easiest way to do that in 2017 is shove them all into a CT log and then spew a list of links to crt.sh or another monitor, rather than uploading some Word document full of X.509 PEM files or whatever. So even CAs that we know don't normally log everything will put all the problematic stuff into logs during disclosure, or else someone reading m.d.s.policy will do it for them.
Some CAs have a deliberate customer policy of letting you choose NOT to log, sometimes with a warning saying if you don't log this stuff then Google might distrust it. Symantec was really into redaction, which is logging but with the "sensitive" bits of the certificate removed. But Google never really bought that idea, and Symantec are exiting the CA business.
Once a certificate is submitted to the logs (which as mentioned elsewhere, Google wants to make mandatory, but today they only enforce for EV and only by treating as non-EV if it isn't logged) the log won't necessarily immediately tell monitors about that certificate. Accepting new items for the logs can be (and usually would be) parallelised, but the log structure is a single Merkle Tree and so cannot be calculated entirely in parallel, usually updating this structure is a serial operation that happens asynchronously, with the result available some time later.
A policy value called the Maximum Merge Delay decides how long a log has to get this done and produce a new head value for the log, Google has currently chosen 24 hours for logs to be accepted in Chrome. Once a log presents somebody with an SCT proving it logged a particular certificate, it has 24 hours to make a Merkle Tree with that cert in it available to the monitors. That sounds like a long time, but it's not "business hours" or "best effort" it's an absolute limit. Data Centre flooded by a tsunami? Too bad. OS upgrade is incompatible with your CPU model? Don't care. Some lunatic threw a billion dollars in bills out of an aeroplane and your employees are out collecting as much as they can carry? Not our problem. Usually you can expect the actual turnaround to be much less than 24 hours, if it's ever greater the log should be distrusted and shut down.
There have already been logs that had "problems" and went away for this reason. Because the logs are only useful if they obey the MMD Google is being pretty strict about this.
However, just because something is in one or more logs doesn't mean any monitor, including famous ones like crt.sh, or Google's own transparency site, has actually obtained and processed the log item right away. There can definitely be slowdowns in this part, but they're only a matter of throwing enough resources at the problem.
NetSol is pretty bad in a lot of ways, but you can make it less bad with some configuration competence. I would have assumed an IT security company would have done that.
> It’s become a widely accepted mantra that experiencing a cyber breach is a question of ‘when’ and not ‘if’.
Part however.
We could say that we will keep seeing more and more hacks.
Numerical solution of implicitly encoded CTMCs
If R is stored as a sparse matrix of columns, a Jacobi or GaussSeidel iteration has the cost of a vector-matrix multiplication, O(n(R))
If R is stored using implicit techniques (Kronecker algebra or decision diagrams), memory is greatly reduced but runtime can increase.
E.g., using Shuffle to compute the effect of v, (pi)^(old) . (operator)(L<k<1) R(k,v1), is slower than Sparse multiplication if the R(k,v) matrices are very sparse.
This research paper describes tracking cattle with a wireless sensor network[1].
---
Or this could be Cisco Technology Migration Program[2]. They could have some sort of network probe to identify "upgrade-ready" products, which could be leveraged for other purposes.
---
[0] http://www.cs.ucr.edu/~cshelton/talks/ctmp-tut.pdf
[1] https://www.researchgate.net/publication/271419298_Cattle_mo...
[2] https://www.cisco.com/web/partners/pr11/incentive/emea/tmp.h...
Basically, they do full packet captures of all network traffic and retain it for a certain period of time so that it can be reviewed later (such as when something like this happens).
CTMp network sensors are servers storing and filtering network traffic, and generating alerts. We use these sensors for our own monitoring services, and provide these sensors to others as part of the Cyber Threat Management platform (software to run your own Security Operations Center, with greater or lesser involvement of Fox-IT experts).
Surprised they don't use one-time passwords, or 2FA exclusively.
https://www.keycdn.com/blog/best-free-dns-hosting-providers/
DNS is REALLY a weak link. Another one is the border gateway protocol. I recently read about all GOOGLE, FB, Apple,etc traffic being redirected through a single Russian address for a few minutes.
So how does a third party get access to the creds to allow messing with your DNS hosting account? Social engineering? Or are there holes we need to know about?