TL;DR: Let's Encrypt certificates.. super convenient, right? Convenient for adversaries, too.
Key segment of TFA:
> Specifically, they were decrypting and re-encrypting traffic as it passed through a network device (the “machine in the middle”) that had been placed between the jabber.ru server and the rest of the internet.
> Usually TLS prevents such an attack from succeeding, as long as you verify certificates. However in this case the attacker was able to obtain valid certificates for the targeted domains, making all connections look like they were genuine.
> With the advent of ACME-based certificate authorities such as Let’s Encrypt, obtaining certificates is not at all hard for someone able to intercept and respond to traffic that is sent to your server, and in this case that’s exactly what happened.
Edit: I'm not saying there's anything wrong with Let's Encrypt. It's just interesting how it made things easier in this instance.
Yes this would be easy and I would expect on most providers it could be done using a snapshot with out affecting the live system for a basic system install unless the file system was encrypted.
Post author here. To clarify, my intention was not to blame ACME for this and certainly not Let's Encrypt specifically (there are many other CAs that will supply certificates via the ACME protocol). I don't consider the pre-ACME way (verification via whois+email) for certificates with a long lifetime to be a more secure or elegant solution than what we have today.
But certainly, the ecosystem is not perfect and it continues to adapt. CAA and (for XMPP at least) channel binding and DANE are the obvious next steps.
I have to ask, do you trust Hetzner? As many mitigations you can take, they control the underlying infrastructure it's a matter of how far they'll go.
Not saying any other provider is better, but this one was caught red-handed. Germany is obviously not the _best_ jurisdiction to host an XMPP server. On the other hand, philosophically, the "good citizen" is the one who abides to the laws of the country they reside.
Not the person you are asking but I trust all legit vendors to follow most of the rules most of the time. All vendors must comply with lawful orders or risk all their equipment being seized. There have been a number of cases were vendors tried to ignore court orders but the most notable one I recall is Lavabit. [1] That predictably ended poorly for them. We won't hear the details of this case until the investigations are tied off.
At best one could use Monero to rent space from one of the shady resellers sometimes referred to as bullet proof hosting but those do not last long and the ones that do last are honeypots. Even then people would have to create their own TLD's, CA's and use /etc/hosts which won't happen due to friction and a lack of discipline. Following that would be private VPN meshes and ephemeral nodes that hide the final hops but those only require a small number of weak links or people to fail as they have in the past.
In summary: I trust them no less than practically any other reputable provider.
If you look at the turbulent history of providers that won't follow court orders, it's a real bucket of fun. Not one I particularly feel comfortable building a sustainable open-source not-for-profit business on top of. Nevertheless, freedom is the heart of what we do: all our software is open-source and easily self-hostable, so if people want to go for those kinds of providers, they're absolutely able to. In fact if you're super concerned about physical security, I can only recommend hosting at home as a solution.
But that doesn't mean we're happy with what happened here. Regardless or who performed it or why, this was an attack on an essential layer of what keeps our systems secure, and that's why we're taking steps to ensure they become impossible. It's also our goal to diversify our hosting locations, to other providers and geographic regions for a number of reasons. That just hasn't happened yet, because we're still pretty small. If you have suggestions of other providers that we should be looking at, I'm very interested!
> With the advent of ACME-based certificate authorities such as Let’s Encrypt, obtaining certificates is not at all hard for someone able to intercept and respond to traffic that is sent to your server, and in this case that’s exactly what happened.
This was easy even before ACME/Let's Encrypt, as several CAs would validate domains using HTTP-based methods similar to ACME's HTTP challenge. These methods were usually less secure than ACME, as CAs would send HTTP requests to non-privileged ports, or use URLs that weren't under .well-known, or allow a validation of www.example.com to be used to issue certificates for example.com.
ACME did not make anything less secure or more convenient for attackers. Rather, ACME raised the bar.
I don't remember putting anything in my web address for domain validated TLS, i could be mistaken.
I have definitely seen email as part of the verification chain and alteration of DNS records: both of which would be hard(er) than intercepting a http request on an infrastructure provider of the host providing the website.
Of course; that would mean the provider who has your DNS could do it, but those are often separate entities and they would have to collude
https://www.devever.net/~hl/xmpp-incident advises "Add support for enforcing the presence of CT proofs (known as Signed Certificate Timestamps (SCTs)) in TLS certificates, and enable this support by default.".
It does not, and we don't plan to push forward on that, at least at this time.
The thing is, although it is the focus of that article, it would not actually have prevented this attack (the MITM certificate was CT-logged and had a valid SCT extension). SCT checking only makes sense in combination with a CAA record and the assumption that no CA could be legally compelled to issue a CT-logged certificate regardless of the CAA record. As far as I'm aware there is currently no way for the CA to prove the CAA existed (or did not exist) at the time of issuance (I'd be happy for someone to correct me if this is wrong, it feels like a big gap in the model).
Channel binding provides far better security guarantees, and does not depend on any third party. That's our focus for the upcoming release.
They do, yes. It's certainly a requirement if channel binding is to work at all.
Additionally there is this proposal to also detect attempted downgrade of the channel binding and SASL mechanism lists themselves: https://xmpp.org/extensions/xep-0474.html - which we're currently looking for expert eyes on, if you know any... :)
One thing this article doesn't mention is monitoring your CAA record too. Unless your DNS is hosted and administrated by yourself, which is unlikely if you are using a hosting service for your jabber service, a targeted attack coordinated by a state/police could also affect your DNS zone records.
Another thing this article doesn't mention is given the nature of the protocol, any user should make sure he communicates to recipients using servers who have set a decent level of security standards. That is the hardest part to do.
22 comments
[ 3.3 ms ] story [ 29.5 ms ] threadKey segment of TFA:
> Specifically, they were decrypting and re-encrypting traffic as it passed through a network device (the “machine in the middle”) that had been placed between the jabber.ru server and the rest of the internet.
> Usually TLS prevents such an attack from succeeding, as long as you verify certificates. However in this case the attacker was able to obtain valid certificates for the targeted domains, making all connections look like they were genuine.
> With the advent of ACME-based certificate authorities such as Let’s Encrypt, obtaining certificates is not at all hard for someone able to intercept and respond to traffic that is sent to your server, and in this case that’s exactly what happened.
Edit: I'm not saying there's anything wrong with Let's Encrypt. It's just interesting how it made things easier in this instance.
Unfortunately, it seems this topic has attracted the attention of many who keep insisting that it must be Let's Encrypt at fault here.
But certainly, the ecosystem is not perfect and it continues to adapt. CAA and (for XMPP at least) channel binding and DANE are the obvious next steps.
Not saying any other provider is better, but this one was caught red-handed. Germany is obviously not the _best_ jurisdiction to host an XMPP server. On the other hand, philosophically, the "good citizen" is the one who abides to the laws of the country they reside.
Not the person you are asking but I trust all legit vendors to follow most of the rules most of the time. All vendors must comply with lawful orders or risk all their equipment being seized. There have been a number of cases were vendors tried to ignore court orders but the most notable one I recall is Lavabit. [1] That predictably ended poorly for them. We won't hear the details of this case until the investigations are tied off.
At best one could use Monero to rent space from one of the shady resellers sometimes referred to as bullet proof hosting but those do not last long and the ones that do last are honeypots. Even then people would have to create their own TLD's, CA's and use /etc/hosts which won't happen due to friction and a lack of discipline. Following that would be private VPN meshes and ephemeral nodes that hide the final hops but those only require a small number of weak links or people to fail as they have in the past.
[1] - https://www.theguardian.com/world/2013/oct/03/lavabit-ladar-...
If you look at the turbulent history of providers that won't follow court orders, it's a real bucket of fun. Not one I particularly feel comfortable building a sustainable open-source not-for-profit business on top of. Nevertheless, freedom is the heart of what we do: all our software is open-source and easily self-hostable, so if people want to go for those kinds of providers, they're absolutely able to. In fact if you're super concerned about physical security, I can only recommend hosting at home as a solution.
But that doesn't mean we're happy with what happened here. Regardless or who performed it or why, this was an attack on an essential layer of what keeps our systems secure, and that's why we're taking steps to ensure they become impossible. It's also our goal to diversify our hosting locations, to other providers and geographic regions for a number of reasons. That just hasn't happened yet, because we're still pretty small. If you have suggestions of other providers that we should be looking at, I'm very interested!
This was easy even before ACME/Let's Encrypt, as several CAs would validate domains using HTTP-based methods similar to ACME's HTTP challenge. These methods were usually less secure than ACME, as CAs would send HTTP requests to non-privileged ports, or use URLs that weren't under .well-known, or allow a validation of www.example.com to be used to issue certificates for example.com.
ACME did not make anything less secure or more convenient for attackers. Rather, ACME raised the bar.
I have definitely seen email as part of the verification chain and alteration of DNS records: both of which would be hard(er) than intercepting a http request on an infrastructure provider of the host providing the website.
Of course; that would mean the provider who has your DNS could do it, but those are often separate entities and they would have to collude
https://www.devever.net/~hl/xmpp-incident advises "Add support for enforcing the presence of CT proofs (known as Signed Certificate Timestamps (SCTs)) in TLS certificates, and enable this support by default.".
Does Snikket do this?
The thing is, although it is the focus of that article, it would not actually have prevented this attack (the MITM certificate was CT-logged and had a valid SCT extension). SCT checking only makes sense in combination with a CAA record and the assumption that no CA could be legally compelled to issue a CT-logged certificate regardless of the CAA record. As far as I'm aware there is currently no way for the CA to prove the CAA existed (or did not exist) at the time of issuance (I'd be happy for someone to correct me if this is wrong, it feels like a big gap in the model).
Channel binding provides far better security guarantees, and does not depend on any third party. That's our focus for the upcoming release.
Edited to add this related discussion a few days ago: https://news.ycombinator.com/item?id=37958552
Additionally there is this proposal to also detect attempted downgrade of the channel binding and SASL mechanism lists themselves: https://xmpp.org/extensions/xep-0474.html - which we're currently looking for expert eyes on, if you know any... :)
Another thing this article doesn't mention is given the nature of the protocol, any user should make sure he communicates to recipients using servers who have set a decent level of security standards. That is the hardest part to do.