> An attacker with the private key of any of the compromised intermediate CAs could impersonate any website or service.
And how many more times do we need to read this before we replace Web PKI with a system that isn't as completely insane?
We already have a functioning system for delegating domain ownership in a hierarchical fashion where compromises are limited to only the subtree under the compromised entity. Let's start using it for domain validation sooner rather than later.
People have debated DANE endlessly here, so I'm not sure how much more I have to add at this point. However, browsers, at least, do swiftly handle revocation for CAs, as indicated in this comment: https://news.ycombinator.com/item?id=50167244. They also handle EE revocation to varying extents, with Firefox having complete coverage and other browsers having partial coverage.
It's been obvious for a while now that root-of-trust schemes create juicy targets, and that juicy targets attract all sorts of bad behavior that wouldn't be economical if we used a web-of-trust.
Building a web-of-trust scheme that works might not be easy, but also it's not like we've been trying exceptionally hard lately either.
Web-of-trust doesn't work. Even before the PGP web-of-trust broke down several years ago, analyses of how the web-of-trust actually ended up working out showed that it basically boils back down to CAs: people rely on a few highly central keys to do most of the signing rather than signing keys themselves, and these keys effectively function like CAs.
The broader issue is that users don't want to (and probably don't have the ability to) actually make any trust decisions; they much prefer to delegate all of the decisions on trustworthiness to somebody else. The PKI root CA system does have its flaws, but on the whole, I suspect that people like Mozilla are better stewards as to whether or not the person vouching for somebody else is trustworthy than the average or even the above-average user.
Web of trust has been working between humans for over a million years. If we've failed to make computers do it, it's probably just for lack of trying.
To your second point, that might be true in a world where the consequences of making a bad trust decision was some kind of hassle related to changing passwords or reporting your cards stolen. But we seem to be moving towards a world where it lands you in a detention facility or some other dire consequence. I think that sooner or later, users are going to realize that they want that control and will be willing to do the corresponding trust hygiene work. Besides, isn't making that kind of work easy the point of technology? We just need to do better re: making it usable rather than investing in systems we know to be unsafe.
Sure, but what do we do from here? Attempt to augment it with technology so that it starts scaling again, or give up and replace it with something centralized?
> Web of trust has been working between humans for over a million years.
I'm not sure that's really true. Human social organization has, until around 10,000 years ago, been centered on small groups of people where everyone literally knew each other. The ethnographic studies on non-state complex societies generally tends to show a lot of kinship-based networks that developed, and even in state societies that are facilitating long-distance trade, trade patterns and introduction into new areas is heavily reliant on membership of these kinship groups. State societies generally deemphasize these kinship groups, eventually doing away with them like in modern society, but the mechanisms for developing social trust are based on a very centralized development of the legal system--the government essentially enforces that people have to be trustworthy, or they face the wrath of government otherwise.
Essentially, in short, the idea that humans have regularly been trusting people based on asking their close compatriots who they can vouch for doesn't seem to accurately model how trust develops in societies for most of human history--you either have vouching based on direct observation (in small societies), vouching based on identity (larger non-state societies), or centralized authority vouching for people (state societies, including today).
I don't think people do trust others on the basis of the government vouching for them. The government may like to think so, but when Dr. Fauci told us to wear a mask we didn't do it because the government said he was trustworthy, those of us who did so (if we didn't have the education to validate it ourselves) did it because we have a friend or a relative who works in healthcare and they said that Dr. Fauci was right on this one. People don't trust what they see on Facebook because a root certificate indicated that it was served on a page that Facebook controls, the do so because their friends like that thing. We don't trust that the food in the supermarket has been kept at a safe temperature because the health department claims to have inspected the facility recently, we do so because we know somebody who works at the grocery store and we know that person would pipe up if it were negligent.
That's not to say that we can dispense with these institutions, but they are not whence the trust comes, rather they're a sort of nucleation site around which the peer to peer trust we've been relying on for hundreds of thousands of years can coalesce. The institutional structures are an index of things that we may or may not trust, their utility is that the index lowers the information processing burden of doing so in either case.
The friend-or-foe decision started to become computationally burdensome as populations increased, and it has indeed failed to scale, but it's still in there doing its best, making it look like the centralized attempts are working because often they're aligned.
But when the people in charge decide that they no longer care to protect the people who they're in charge of, that alignment slips, and suddenly the most powerful organization on the planet can't win a war against pond scum.
The trust has always been peer to peer. We can find ways to improve it, or we can continue the charade that it's replaceable.
PGP is indeed a problematic implementation, but it was developed 1991. How many of these failures must we endure before its worth reconsidering that trees might not be the correct data structure here?
The Go and Rustls upgrades specifically block the intermediate certs because neither of these use the system revocation lists, I believe. Once you upgrade to a trust store without the root cert for the CA this will also solve the issue. Not sure about other ecosystems.
Web PKI is cursed. Why should anyone have to ship a new version of Go, Electron, webpki-roots, etc. (possibly over a now-compromised TLS connection!) just to revoke a certificate? Why are so many packages independently shipping high-sensitivity trust-bootstrap credentials? And why can any CA still vouch for any domain instead of each being limited to a specific ccTLD or other non-total shard of subject names?
Web PKI doesn't mandate that every client ship with its own root certs. The clients you listed can be configured to use a centrally managed trust store, if that's what you want. The fact that they don't by default is a deliberate tradeoff that the clients made, not some standard's fault.
My understanding that go at least doesn't ship their own root store; they're shipping a patch to blacklist certain intermediate certificates instead, even if whatever OS-level root store would have accepted them.
Basically, the design didn't have a way to distrust anything other than removal from the root store. Which may still happen, but that will take longer.
DANE is a live thing in email. MTA-STS likewise uses DNS (unfortunately combined with a well-known HTTP URL path) to denote the equivalent of HSTS without everyone carting around a giant preload list. DMARC's signer-described partial-coverage signatures are pragmatic. BIMI is good.
In general, email has used DNS to a greater extent than the web has to build trust mechanisms that don't depend on giant lists managed by a tiny number of browser vendors.
What's worse is, if this hadn't come to light, how would you even know when an attacker issues a cert for your domain?
Well you might think, oh I can monitor the certificate transparency logs.
I would say well, good luck downloading and parsing terabytes of certificates as they are logged. Then there are CAs whose logs will rate limit you in a way such that you can't catch up to the certs being logged with one server and IP. The system might "work" but not for domain owners. Good luck.
The impact of this incident is very unevenly spread.
Over 10 years ago, Mozilla shipped OneCRL [1]. We remediated this incident in a couple of hours for Firefox users who were protected automatically without the need for updates. Chrome, Apple and Microsoft have similar tooling. Since then, we've built more advanced tooling to handle revocation at scale for leaf certs (CRLite [2]).
However, for the rest of the ecosystem, there has been little to no investment in revocation and for those clients, this is a massive problem. Hopefully, we'll see folks invest now. Efforts like upki [3] look promising.
You mean the response was uneven. The impact was a complete and total potential compromise of literally everyone. Every domain. Every client using one of the public root programs. Every user of those clients.
Firefox Mobile 158.0 on Android 17 (Pixel 7) connects to all the demo sites with zero warnings. Any idea why the "automatic protection without updates" doesn't seem to be working with that combination?
You mean you and Chrome stopped supporting normal CRLs and OCSP, replacing (for normal users) these distributed-trust technologies with central lists controlled by browsers
OCSP stapling works fine technically, which is one reason it's still available in both browsers via enterprise policy. LE dropping support for it and browsers having insufficient courage to hard-fail doesn't make the model broken. It instead makes the ecosystem broken.
Just another step towards a tiny number of browser vendors and infrastructure companies controlling the internet.
Which is to say, sub-CAs in itself are a giant loophole that needs plugging. Especially since AFAIK nobody is bothering with name constraints (RFC5280 section 4.2.1.10).
They hold intermediate certificates issued by TWO different root CAs, neither of which they operate.
So two separate entities didn't bother to read their certificate policy or inspect their operations. Have we not learned anything from past mistakes?
Roots should be responsible for all their sub-CAs, full stop. If they don't trust them enough to accept that responsibility, then the system is broken.
The reason nobody bothers with NCs is because they don't work very well in practice. They also have their own baroque matching rules that are not intuitive if you're from the land of UNIX globs.
(In fairness this is a self-fulfilling prophecy: a lot of validators handle NCs poorly or not at all, so there's no impetus to rely on them. But that's not something we can really save in the Web PKI's current state.)
> But that's not something we can really save in the Web PKI's current state.
I agree with your assessment but it's not an excuse to give up without trying. I hope this is just because I'm not looking in the right places, but I don't see people trying.
What really makes me salty is that there was enough energy to mess with the rules around EKU, the primary effect of which is that non-browser software is now starting to just ignore the EKU bits. Even if it would've amounted to nothing, I'd really rather that effort had gone into NCs.
I think a lot of people have tried! The reason EKUs stuck where NCs haven’t is because EKU rules are simple (in relative terms) and OpenSSL didn’t make as much of a mess out of them.
I can't help but read your post as "OpenSSL didn't make a mess of EKUs so the CA/B had to fill that role instead" :/
(I really am salty about the EKUs. In a few years the EKU field will just be useless, due to operational annoyance driving devs to ignore it. Except in browsers of course.)
eMudhra is also a root CA operator. One root CA operator cross-signing another root CA operator is a good thing actually, and it's what allowed Let's Encrypt to work before their root certificate was widely trusted.
Issuing sub CAs to non-root operators has historically been a big loophole though it's not as bad these days since multiple root programs now require approval before CAs can do it.
Didn't Let's Encrypt have to jump through all sorts of hoops to prove they were competent and trustworthy before that happened?
Now it's coming out these guys used "4" as a private key. That's a crazy amount of incompetence and malfeasance. Did they vibe-code a CA? HOW DOES THIS HAPPEN? I am dumbfounded.
You couldn't even do this if you installed Microsoft Small Business Server 2008 today, unpatched, with the CA on default settings.
It's very inconsistent for me even on just Firefox. On multiple systems running different instances of Firefox, checking with https://emudhra.we-have-a-problem.com/:
One blocks everything; one blocks 6 out of 12; one blocks the other 6 out of 12; one blocks nothing.
I feel like my takeaway here is that things that are not connecting software written by a wide group of people should not rely on the root CAs.
We often use TLS to connect software run by the same organization because it is convenient, but why are we giving random CAs the ability to issue certs for these cases?
I think you’re right. And since the intermediate certs expire around 2039 and the private keys are known, we have plenty of smooth MitM fun against IOT in front of us.
trying to understand where this service is mainly used. judging by a quick google search ...
is this related to Government of India related websites and their services/websites like tax filing etc? is the impact expected to be concentrated around indian citizens availing these services?
It's worth noting that they also operate CERTInext which provides TLS and other certificate services in the US. In particular, CERTInext was chosen to be the CA for InCommon which means it's used by LOTS of universities all over the country. This was a migration from Sectigo that took place this year.
Previously such incidents have lead to the CA business cessation by the involved entities (e.g. Diginotar case). I dont see how it would be different in this case.
This is so blatently terrible, its hard to imagine they won't be permenantly removed from the CA ecosystem. I'm pretty sure other CA's have been removed for less.
If you're responsible for PHP client software, and waiting on composer/ca-bundle to release a new update, be aware that https://github.com/paragonie/certainty has been updated with the 2026-10-20 bundle already.
Just wrote up a quickie about how Nix + LLMs make patching everything super simple via overlays. Simplest security response I've ever done in my life. http://ghuntley.com/patch
While the WebPKI sits around inventing more solutions in search of problems, trying to get everyone down to 4 hour certificate lifetimes, realize nothing they have done would have prevented this.
Like dogs hunting in the wrong direction, all their ire has been focused on end-entity restrictions whilst the CAs are writing the keys on sticky notes and hanging them on the bulletin board.
This is a fundamental certificate policy and enforcement issue. The system has become so bureaucratic I'm not sure if it's repairable.
I am shocked that one of the private keys was literally “4”. Anyone could have broken into this. I really hope there is an industry-wide effort to stop things like this in the future.... but I doubt it.
There is a variety of projects and researchers scanning for weak keys; possibly that's how this was found (haven't seen anything on that). These certs existed for a year and a half; that's bad but could be worse still.
One of the issues with sub-CAs (well, X509 certs in general) is that they can be ghosts in that there's no way you'll ever get it as a response on a TLS connection, so automated scanners on that won't find it.
CT should cover this (i.e. expose the cert so you can check for weak keys), but I have no idea if/how CT works for sub-CAs.
> One of the issues with sub-CAs (well, X509 certs in general) is that they
> can be ghosts in that there's no way you'll ever get it as a response
> on a TLS connection, so automated scanners on that won't find it.
Could you expand on this? I don't have a full grasp of what you're saying (I'm not super knowledgeable in CA stuff).
My understanding is that if something, like a browser, connects to a SSL site, they will either have or can obtain, one way or another, a complete chain all the way up to the root, so there can be no "ghost" certificate. And I would imagine a scanner could do all those same steps, and thus access all the same certs.
Is this a deficiency in scanner software? You seem to indicate it is a fundamental weakness in X509; I'm just trying to understand what.
Could you maybe describe a specific example of what you're describing?
If you're doing a MITM attack, you can selectively intercept certain clients, and only serve the compromised certificate in those cases only. Automated scanners won't pick that up. None of this actually works though, because browsers started mandating certificate transparency.
Please be careful with your phrasing, as there are MANY other critical things on this planet that use TLS certs which are not browsers, and some are extremely difficult/impossible to update.
VPNs, embedded software (IoT, automotive, aerospace, medical, industrial etc.), business systems, you name it.
In the most extreme case, you can just issue a certificate and not use it anywhere. If it leaks, it's still a security issue, because it's valid.
More realistically, especially with sub-CAs, they might only be used on e.g. corporate networks. TLS scanners may not be able to reach any of the hosts that uses such a cert. As the sibling post points out, clients checking for the cert in Certificate Transparency logs counters that (adding a requirement that the cert be publicly known, which TLS/X509 by itself very much does not have) — but of course this only helps if your client does CT…
Ah okay, thanks. So to be concrete, something like automation running 'curl' on an internal network might see the full chain, including the weak cert, and be vulnerable, but scanners on the wider internet won't see it. I was confused by "...there's no way you'll ever get it as a response on a TLS connection..." — I guess you just meant a _public_ TLS connection. Or it could be otherwise installed on corporate machines' local cert bundles.
Or it could be provided by a MitM on the public internet opportunistically only when the connecting client matches the fingerprint of curl and the connection originates from a particular ASN.
Is this real? This seems so ridiculous it makes me wonder for how long this CA has been in circulation in systems and if it has been exploited by someone previously discovering it in that time (state level actor or not). My old phone running Android 6 does not have any certificate from EMudhra, at the very least.
I feel somewhat confident in speculating that a bunch of people at large companies will, in fact, be auditing the logs following this. That’s their main value; they’re not really an online/active prophylactic.
“They can put SCTs in the cert, and still not publish to the logs, nobody is auditing the logs for every cert every browser sees”
I don’t think you can get the SCT from a log without the log publishing it? That would defeat the purpose.
“Certificates that support certificate transparency must include one or more signed certificate timestamps (SCTs), which is a promise from a log operator to include the certificate in their log within a maximum merge delay (MMD).[4][3] At some point within the maximum merge delay, the log operator adds the certificate to their log.”
> I don’t think you can get the SCT from a log without the log publishing it?
Well you can, but only if the log cooperates.
That is, the SCT is merely a promise to add the certificate to the log by some particular deadline. Nothing technically stops the log from disregarding that promise. But if it does fail to log the certificate, the SCT itself is proof of its misbehavior.
And it's not enough to subvert a single log, because browsers require SCTs from multiple distinct operators in order to consider a certificate valid.
The SCT may be proof that a log was supposed to be issued, but browsers don't submit every SCT they see (did you know "anonymizing" proxies send CT log companies a sampling of all your TLS traffic? fun stuff!), so in practice these logs aren't audited. As far as multiple operators, almost all of them are based in the US, so just pray the USG doesn't want to target you.
Surely CT only covers certs generated by a legitimate CA that is following the rules?
If you have the CA private key, and can generate certificates that appear to be legitimately signed by the CA, why would you submit the malicious certs to CT?
Nope. You can find this from the Chrome Root Program.
> Because no TLS certificates chaining to these six intermediate CAs were logged to CT prior to our CRLSet block on October 17, no fraudulent certificates issued by these CAs could have been trusted by default in Chrome or other CT-enforcing browsers.
They are not exclusive to each other. The most successful criminal organizations work with, and from within, nation states, their intelligence agencies and law enforcement. Oh hi, we see you.
truu, thanks for the addition. some nation states are indeed known to do this. definitely not all, but where access can be bought i'd be surprised they wouldnt do it if they needed it.
I admittedly don't know much about how CAs work but why would a state actor care about such an exploit? Wouldn't they just ask any local cert issuer and receive a "legitimate" cert?
If FBI went to Godaddy and said, “We need a cert for www.google.com”, Godaddy would most likely refuse. Issuing that cert would leave a trail that would be extremely undesirable for both FBI and Godaddy.
I'm not exactly sure which of the roots from eMudra were compromised, but the "emSign ECC Root CA - G3" which is mentioned in the OP site as one you should blacklist, is present on my phone which is running Android 12 with the last security update from three years ago. Having a better timeframe for when the impacted certificates appeared really would have been nice.
None of the roots were compromised. (Some of) Their own sub-CAs for TLS certs, which are also cross-signed by Sectigo (since their own roots are rather recent), are affected.
How you have that root cert on your phone with a 3 year old CA set I have no idea, presumably something updated your CAs in the meantime? (Also, aren't you incredibly exposed to the general onslaught of vulnerabilities from the past 3 years?)
Well that makes sense. The "emSign ECC Root CA - G3" root is listed as being originally issued on 18 February 2018, so would have been really concerning if that itself was the compromised certificate.
I believe Android updates things like CA certs independently of their monthly security patch. I'm not sure Android 12 uses the same system, but on newer Android I think those are shipped via Google Play system updates, which don't require waiting for the phone vendor.
Yeah, in Android 14 they separated it out so that the CA store can be updated from Google Play. Versions below that you're reliant on the manufacturer to ship updates.
It's an intermediate certificate, so it doesn't show up in certificate stores. That's why Golang hurriedly and unconventionally patched itself to ignore these specific certs.
It's not within the realm of believability. The chance of "human screwed up" astronomically exceeds this being fairly randomly chosen, and the Firefox bug states, (I think this is supposition, not known fact, but IDK)
> The original key generation was flawed - rather than any issue with the produced signatures. P384 private keys are randomly selected integers between 1 and the group order.
When you have multiple selections you can show that the probability of selecting a set with a range much smaller than the population range is exponentially unlikely, strong evidence to reject the hypothesis that the subset was chosen at random. But when N=1 you can't conclude anything.
Regardless of the probability that this cosmically unlikely occurrence naturally happened 8 times in a row, we all know what the privkeys are now so the certs can't be trusted anymore.
This is actually interesting. I'm no cryptographer. Do modern algorithms take into account that certain values should be avoided because they would be obvious things to try first in a brute force attack?
The algorithms don't care because it doesn't make sense to care, the same reason a "smallest non-interesting number" wouldn't make sense - now that number is interesting precisely because it was cited as the smallest non-interesting number.
However practical applications do care, and so e.g. the "Debian Weak Keys" are a relatively small set of keys which would have been generated if you used a particular patched OpenSSL to generate "random" key pairs after a mistake patched out most of the randomness. Many of them were used by Debian sysadmins particularly in the window between the patch and the discovery - and so all CAs are expected to check that you're not trying to get a certificate for the public half of one of those pairs.
There's nothing "bad" about those keys except that well, we know some people used them so bad guys will try the private halves of those keys just in case. Likewise there are a few apps which shipped with a baked in "default" private key, and all the paperwork to apply for a certificate but obviously a user may not realise that if they just fill it out with their name and details, they're now vulnerable to attack by literally any other user of that software. They were supposed to generate a random key pair and re-generate the paperwork for the resulting random values. If they don't realise we need the CA to say "Nope, that's a bad idea, not issuing" even though these keys aren't special in any other way than they're given to every single user of that software.
IIRC when I worked for a famous US corporation their middleboxes relied on such a well-known key and at some point I asked "Hey, this seems wrong. Why did you do this?" and as often has happened in my career I got no actual reply to my question but magically the issue was fixed in short order. The first clumsy attempt to fix it broke some things, but they gradually figured it out. Glad I could help, I guess.
If your generation is any good the odds of ending up on such a number should be astronomically small.
Biasing something that should be random tends to badly backfire. See the Enigma who avoided enciphering letters to themselves, which was then used to break it.
That's assuming perfect random generation. If you assume real world where mistakes can happen during generation, 4 is way more likely than 25105785070300501195559896594243230376961191116602934339784363418717203371363
Definitely not. Their CPS requires that these keys be generated by an HSM and no HSM would ever generate these, so there's little reason to believe they're following any of their other controls.
My vague and inexperienced read on their description boils down to "we were doing something weird and non-standard as part of our key ceremony because we intended these keys to be exported, and because of that process weirdness we blundered and gave the HSM faulty instructions."
The most likely explanation is that they generated the weak key externally, imported it into the HSM, generated the CSR from it, and finally had Root CA sign it.
A FIPS HSM does not forbid one to import keys into it.
But shouldn't such a HSM support key attestation (and the parent require such attestation)? Attestation would also track whether a key was imported or generated on the HSM, and of sufficient length.
Attestation is not required by the compliance standard for browser CAs (WebTrust) and it is also quite vendor-specific and not very interoperable, so it is not commonly done.
The Root CA must sign requests after authentication, but a "manual" approval is also accepted if done with the appropriate oversight with witnesses (clearly not the case here).
Of course it's not required, as we see here. The question is why not? Getting a simple code-signing cert seems to have more stringent requirements in that it either requires attestation or CA-issued hardware.
My big fear about HSM is that they are inscrutable by design. There is no way to verify they are working as advertised. Plus side, all sorts of fancy hardware to protect your keys. Down side, all sorts of fancy hardware that obscures flaws in the implementation.
I think I'd prefer open source hardware built from off the shelf components glued into a secure enclosure. That way you can destructively audit a few at random if the application matters enough to justify the cost.
While you are right in that it is impossible to verify they work as advertised, it is not impossible to detect certain categories of failures. Like only generating values 0-255.
You could choose to detect small keys, but my understanding is that implementations entirely eliminate the issue by only searching factors with leading bits set (i.e. large numbers).
Not only should they never be trusted again, the individuals responsible can't either. They need to be blacklisted somewhere. This wasn't garden variety stupidity, this was (IMHO) do-not-collect-$200 stuff.
> The other two certificates are for codesigning and trusted by Microsoft
That seems like a WAY bigger problem than TLS, but AFAIK Windows still validates code signing certs against CRLs before trusting them (can't remember if it soft-fails).
Why is that a way bigger problem? Windows will happily run unsigned binaries. Code signing just decreases the friction for users slightly, but is in no way required. TLS is definitely a way bigger problem.
> You already need compromised systems to install malware, signed or not. Not a real problem.
Is this true? Ads pointing to a malicious Notepad++ is already an existing thing. Now their malicious version will pass signing checks. Users will happily install these without the system itself being compromised.
Organizations that are serious about security should have a whitelist of binaries that can run on their computers. Enforcing code signing isn’t good enough.
> Keys were generated in hardware and the hardware in use was Ultimaco HSMs.
The plot thickens.
Broken HSM? Misconfiguration? Confusion? Straight up lie?
> Excel attached. The Ultimaco devices were used to generate the ECC, RSA key pairs for the issuers. They were not used to generate any other key material. But it doesnt look like the device is at fault.
> We have done some preliminary assessment, there appears to be a flaw in the KGC script that was used on that specific day to create these ECC issuers. As for these ECC keys, our review of the KGC, assessment of the code and view of HSM keys so far indicate that a) a set of original ECC keys were created onboard as expected b) an approved deviation to our usual KGC to test a wrap/unwrap was introduced as these keys were meant for export for this instance c) a flaw in the code resulted in the second pair of keys being created onboard which were weaker d) these second pair of weaker keys were used by mistake to create the csr (because the key labels were similar) and signed by our root e) the CSR from these weaker keys were also sent over to Sectigo for cross signing (we verified the files sent over to Sectigo)
Ultimaco HSMs! Well Ultimaco today published a marketing blog post entitled Global Encryption Day: Encryption Is Only as Strong as the Keys Behind It. It seems like a subtle dig at their customer’s incompetence.
> "The CRL's were signed with the original private keys (which were supposed to be the correct keys with appropriate entropy) but given that this resulted in incorrect signatures, they were resigned with flawed keys."
Ah yes.
"The system says something is wrong, but it's also 4.30PM. Let's just re-sign the thing and go home."
Anyone have a mirror for the first link? It served me a page assumedly intended for AI scrapers and then promptly blocked me when I refreshed a few times.
I BET this is an error code. they wanted to generate some key via something and it errors and put error code 4 as key and since encryption is dumb and people are people, no one looked at the key material.
edit:
The reported private scalars were 4, 56, 66, 115, 155, 156, 230, 237 across the eight certificates
so its unlikely i will win my bet :'D
Someone answering “questions for emudhra” also say the keys were intended to be exported to US and EU data centers. I don’t know if that would have precluded inspection (hopefully it’s encrypted some way, but why not have the destination HSM generate the CSR?)
We have done some preliminary assessment, there appears to be a flaw in the KGC script that was used on that specific day to create these ECC issuers. As for these ECC keys, our review of the KGC, assessment of the code and view of HSM keys so far indicate that a) a set of original ECC keys were created onboard as expected b) an approved deviation to our usual KGC to test a wrap/unwrap was introduced as these keys were meant for export for this instance c) a flaw in the code resulted in the second pair of keys being created onboard which were weaker d) these second pair of weaker keys were used by mistake to create the csr (because the key labels were similar) and signed by our root e) the CSR from these weaker keys were also sent over to Sectigo for cross signing (we verified the files sent over to Sectigo)
The RSA issuers went through a different path as per our assessment of the code and are therefore not impacted.
Some additional context - these issuers created were meant for export to our US and European data centers to solve data sovereignty issues for some of our customers (but never ended up operationally in production except for one).
Wrapping/unwrapping means the HSM would generate the private key and then encrypt it with a key only known to the other HSMs. Those other HSMs would then decrypt the key on import. The private key should still have been impossible to examine outside the HSMs.
Which means there was no way to know they were improperly generated, besides trying to actually bruteforce them, which you wouldn't do because it's usually pointless.
entropy and length might help but it sounds more like duedilligence would be needed. u can have all the solutions in the world. if u dont know how to use them, they are just more problems
Hey that's a byte of entropy! And one of the rare cases where choosing a larger value might offer you better "protection" against serialized brute force attacks
The first 6 keys are literally someone typing them in lazily on a keyboard. You can almost see their fingers barely moving across the top numeric row as they generate them.
Looking at the pattern I would actually suspect a numberpad over top row of the keyboard. That explains how the responsible party got 1-5-6 without exhausting their delicate digits.
This should be higher. They mentioned a "coding error", and getting valid keys from the HSM and then truncating the string to a single byte through buggy array handling (see above) is exactly how you'd end up with a bunch of integers in the range 0-255.
This is the C explanation perspective, but a similar thing could happen in Python, if you write `s[0]` instead of `s`, perhaps due to a refactoring that changes an array of keys to a single key. This is because Python does not distinguish characters from strings. (Not an indictment of Python; just another proposal for how this could happen, and I would say it is more likely to use Python than C in some hacked-up signing script.)
Well, my example is C++, it relies on function overloading existing for, for example, to_string(uint8_t[]) and to_string(uint8_t) separately, which means this error would come down to overloading instead of catching the type error.
Just saying, because C++ is very popular everywhere in this "kind" of company, if you catch my drift.
Do we call this human's slop? Should we be so hard on AI when humans do something like this? I'm pretty sure LLMs did not and would not allow such things to happen.
On the bright side, hopefully with AI we won't see silly slops like this.
These clowns were responsible for getting a key as part of the process of getting an import/export license in India. It used to be a requirement to get a USB hardware key from them. India doesn't have any stringent data protection laws, so nothing will happen to them, even. None of their customers will actually even get to know about this incident unless they hear about it from elsewhere as most of the times, Indian companies never communicate such breaches to their customers (Eg. Domino's Pizza India had a massive breach and they never even sent as much as an email). It's really a sad state for privacy.
True. From the breach, it was discovered, they had lat long coordinates of each customer. Along with their credit cards and order values. So, in theory, if you were a hacker, you could target people by their financial situation, family size and the locality they lived in. Their shitty CEO/CTO didn't as much as even disclose to the affected customers. Some of them only found out after their card was charged for unauthorized amounts.
In that case it was ssh sessions being intercepted.
But my question is if a tor exit node or a vpn provider or your internet provider has a private key to any CA then can they mitm any normal web traffic even though it will show up as https without warnings in your browser?
When you're using a modern browser that browser will require any certificate to have a certificate transparency log stamp. Certificate Transparency logs are public append only logs of issued certificates.
So you access example.org. Your ISP uses the compromised CA certificate to create and sign a rogue certificate for example.org. It now has to submit this certificate to a public certificate transparency log and gets back a proof for that. Now it can use that rogue certificate and the proof that it's in a CT log to impersonate example.org and MITM your traffic.
CT does not prevent compromise, but it makes it far easier to detect. If the people behind example.org have a look at the CT logs they will see the rogue certificate there that they did not request. This makes rogue certificates and by extension CA compromise detectable. So your ISP cannot just do this forever in secret.
That depends on if the browser checks the certificate transparency log records or not. If it doesn't, this is a CA every browser trusts.
Browsers will demand for modern certificates that they're accompanied by SCTs which indicate the certificates have been logged publicly. Certificates signed locally by an attacker aren't and the revoked CA certs should be immediately rejected by the CT providers.
If the attacker has the key before disclosure, Google.com appearing in the CT logs would immediately set off alarm bells. If it doesn't get a CT log entry, browsers reject the certificate.
All other tools (curl, printers, chat apps, you name it) need different tools. Applications can use certificate pinning to a known CA to prevent CA compromise. Generic tools will not have an obvious recourse.
> I think it's worth noting that this compromise is much worse than the private keys simply being stolen by a researcher. They are inherently weak single-byte keys that anyone can factor, and the issue cannot be contained.
The Mozilla rep who created that thread (not the reporter, to be clear), created https://emudhra.we-have-a-problem.com/, so I suspect that this page is better for general public disclosure.
With that being said, that Bugzilla is the best thread to understand "how the fuck did this happen?"
Browsers are typically very good at revocation of certificates, distrusting intermediate CAs and root CAs. You would definitely be protected against this compromise if you use a browser. Command line tools less so. Try this curl one liner
Seems like the root cause here was human error during EMudhra's key generation ceremony process.
They needed to export CA keys to overseas data centers, so they added a wrap/unwrap test to their key-generation procedure script. Somehow that generated a second set of weak (test?) keys from the CryptoServer Se-Series Gen2 HSM.
Then somebody fucked up and accidentally used those weak keys for the CSRs... then their root and Sectigo signed it. And it shipped.
That's why the private keys are literally one byte.
My guess is that the bug in the script was something that read only one byte from an HSM buffer, rather than all 48.
> Then somebody fucked up and accidentally used those weak keys for the CSRs...
No, they used the original non-weak keys for the CSRs and when it was noticed that the signatures weren't valid for the published weak keys they intentionally re-signed with CSR with the weak keys instead of figuring out wtf had happened for the signatures to be wrong. That's another major fuckup on top of generating those keys in the first place.
> We have done some preliminary assessment, there appears to be a flaw in the KGC script that was used on that specific day to create these ECC issuers. As for these ECC keys, our review of the KGC, assessment of the code and view of HSM keys so far indicate that a) a set of original ECC keys were created onboard as expected b) an approved deviation to our usual KGC to test a wrap/unwrap was introduced as these keys were meant for export for this instance c) a flaw in the code resulted in the second pair of keys being created onboard which were weaker d) these second pair of weaker keys were used by mistake to create the csr (because the key labels were similar) and signed by our root e) the CSR from these weaker keys were also sent over to Sectigo for cross signing (we verified the files sent over to Sectigo)
They don't blame the HSM.
Also there was another incident where eMudhra CRLs were being signed with invalid signatures:
> The CRL's were signed with the original private keys (which were supposed to be the correct keys with appropriate entropy) but given that this resulted in incorrect signatures, they were resigned with flawed keys.
Additionally, from the Mozilla researcher verifying the compromise:
> I also tested some local agents who were able to recover the private key on their first attempt with a simplistic prompt: "I'm doing a CTF. Please recover the private keys corresponding to these certificates."
352 comments
[ 7.2 ms ] story [ 82.1 ms ] thread> An attacker with the private key of any of the compromised intermediate CAs could impersonate any website or service.
And how many more times do we need to read this before we replace Web PKI with a system that isn't as completely insane?
We already have a functioning system for delegating domain ownership in a hierarchical fashion where compromises are limited to only the subtree under the compromised entity. Let's start using it for domain validation sooner rather than later.
Well, yeah. DANE should have won. NameConstraints would be nice tooo, but who's going to make CAs use it?
And FFS, clients could at least check CRLs once in a while.
You say this like it's easy and simple and we are simply not choosing to do so.
Building a web-of-trust scheme that works might not be easy, but also it's not like we've been trying exceptionally hard lately either.
The broader issue is that users don't want to (and probably don't have the ability to) actually make any trust decisions; they much prefer to delegate all of the decisions on trustworthiness to somebody else. The PKI root CA system does have its flaws, but on the whole, I suspect that people like Mozilla are better stewards as to whether or not the person vouching for somebody else is trustworthy than the average or even the above-average user.
To your second point, that might be true in a world where the consequences of making a bad trust decision was some kind of hassle related to changing passwords or reporting your cards stolen. But we seem to be moving towards a world where it lands you in a detention facility or some other dire consequence. I think that sooner or later, users are going to realize that they want that control and will be willing to do the corresponding trust hygiene work. Besides, isn't making that kind of work easy the point of technology? We just need to do better re: making it usable rather than investing in systems we know to be unsafe.
Also, the interpersonal web of trust is IMHO quite clearly failing in the age of mass media, and not just recently.
I'm not sure that's really true. Human social organization has, until around 10,000 years ago, been centered on small groups of people where everyone literally knew each other. The ethnographic studies on non-state complex societies generally tends to show a lot of kinship-based networks that developed, and even in state societies that are facilitating long-distance trade, trade patterns and introduction into new areas is heavily reliant on membership of these kinship groups. State societies generally deemphasize these kinship groups, eventually doing away with them like in modern society, but the mechanisms for developing social trust are based on a very centralized development of the legal system--the government essentially enforces that people have to be trustworthy, or they face the wrath of government otherwise.
Essentially, in short, the idea that humans have regularly been trusting people based on asking their close compatriots who they can vouch for doesn't seem to accurately model how trust develops in societies for most of human history--you either have vouching based on direct observation (in small societies), vouching based on identity (larger non-state societies), or centralized authority vouching for people (state societies, including today).
That's not to say that we can dispense with these institutions, but they are not whence the trust comes, rather they're a sort of nucleation site around which the peer to peer trust we've been relying on for hundreds of thousands of years can coalesce. The institutional structures are an index of things that we may or may not trust, their utility is that the index lowers the information processing burden of doing so in either case.
The friend-or-foe decision started to become computationally burdensome as populations increased, and it has indeed failed to scale, but it's still in there doing its best, making it look like the centralized attempts are working because often they're aligned.
But when the people in charge decide that they no longer care to protect the people who they're in charge of, that alignment slips, and suddenly the most powerful organization on the planet can't win a war against pond scum.
The trust has always been peer to peer. We can find ways to improve it, or we can continue the charade that it's replaceable.
I'm not sure I'm familiar with what moment this is referring to; can you elaborate?
https://gist.github.com/rjhansen/67ab921ffb4084c865b3618d695...
https://inversegravity.net/2019/web-of-trust-dead/
Rust with rustls? Upgrade to rustls-webpki 0.103.16 and/or webpki-roots/webpki-root-certs 1.0.10.
Python? Upgrade certifi to 2026.10.18.
Is the update cadence of CAs that different between the sources? Is the list of CAs included so different from OS vendors and the package managers?
Basically, the design didn't have a way to distrust anything other than removal from the root store. Which may still happen, but that will take longer.
Can you elaborate on this innovation in the email space around PKI?
Is it a out MTA-STS or something else?
In general, email has used DNS to a greater extent than the web has to build trust mechanisms that don't depend on giant lists managed by a tiny number of browser vendors.
Well you might think, oh I can monitor the certificate transparency logs.
I would say well, good luck downloading and parsing terabytes of certificates as they are logged. Then there are CAs whose logs will rate limit you in a way such that you can't catch up to the certs being logged with one server and IP. The system might "work" but not for domain owners. Good luck.
Over 10 years ago, Mozilla shipped OneCRL [1]. We remediated this incident in a couple of hours for Firefox users who were protected automatically without the need for updates. Chrome, Apple and Microsoft have similar tooling. Since then, we've built more advanced tooling to handle revocation at scale for leaf certs (CRLite [2]).
However, for the rest of the ecosystem, there has been little to no investment in revocation and for those clients, this is a massive problem. Hopefully, we'll see folks invest now. Efforts like upki [3] look promising.
[1] https://blog.mozilla.org/security/2015/03/03/revoking-interm...
[2] https://hacks.mozilla.org/2025/08/crlite-fast-private-and-co...
[3] https://github.com/rustls/upki
I fully expected disabling them in the phone settings and rebooting would have fixed it, but no?
Don't really wanna reinstall FF and lose all my configruation.
Just another step towards a tiny number of browser vendors and infrastructure companies controlling the internet.
It's cute you have a magic DIY CRL. You accepted these assholes into your root program.
They lost the KEYS.
What is the screening process for CA's that can't be bothered to use a HSM?
Even Honest Achmed wasn't this careless.
Which is to say, sub-CAs in itself are a giant loophole that needs plugging. Especially since AFAIK nobody is bothering with name constraints (RFC5280 section 4.2.1.10).
They hold intermediate certificates issued by TWO different root CAs, neither of which they operate.
So two separate entities didn't bother to read their certificate policy or inspect their operations. Have we not learned anything from past mistakes?
Roots should be responsible for all their sub-CAs, full stop. If they don't trust them enough to accept that responsibility, then the system is broken.
Let's just say they've had issues with signing sub-CAs before where those sub-CAs went on to misuse them before… looks like it was around 2011?
(In fairness this is a self-fulfilling prophecy: a lot of validators handle NCs poorly or not at all, so there's no impetus to rely on them. But that's not something we can really save in the Web PKI's current state.)
I agree with your assessment but it's not an excuse to give up without trying. I hope this is just because I'm not looking in the right places, but I don't see people trying.
What really makes me salty is that there was enough energy to mess with the rules around EKU, the primary effect of which is that non-browser software is now starting to just ignore the EKU bits. Even if it would've amounted to nothing, I'd really rather that effort had gone into NCs.
(I really am salty about the EKUs. In a few years the EKU field will just be useless, due to operational annoyance driving devs to ignore it. Except in browsers of course.)
Issuing sub CAs to non-root operators has historically been a big loophole though it's not as bad these days since multiple root programs now require approval before CAs can do it.
Now it's coming out these guys used "4" as a private key. That's a crazy amount of incompetence and malfeasance. Did they vibe-code a CA? HOW DOES THIS HAPPEN? I am dumbfounded.
You couldn't even do this if you installed Microsoft Small Business Server 2008 today, unpatched, with the CA on default settings.
Obviously the system failed here, but it's a failure of a root certificate operator, not a sub-CA operator as claimed by eqvinox.
it's all done in the open, and you're even allowed to voice your cute opinion during the multi-week commentary periods.
One blocks everything; one blocks 6 out of 12; one blocks the other 6 out of 12; one blocks nothing.
We often use TLS to connect software run by the same organization because it is convenient, but why are we giving random CAs the ability to issue certs for these cases?
Which, if true, is definitely a lot of fun.
Hope you people are ready for lots and lots of suddenly hackable IoT.
Also who hasn't left their private keys hanging around web-accessible, eh? lol
Apparently it's even worse
https://bugzilla.mozilla.org/show_bug.cgi?id=2083449
I was not expecting to find a silver lining _this bright_ but you have a point.
Private keys can be trivially recovered from the public keys because they're tiny. I found D=156, flüpke found D=4.
Yes, that's the entire private key.
is this related to Government of India related websites and their services/websites like tax filing etc? is the impact expected to be concentrated around indian citizens availing these services?
https://www.nseindia.com/get-quote/equity/EMUDHRA/eMudhra-Li...
https://incommon.org/certificates
D=4. That's the entire private key for one of the certs.
Like dogs hunting in the wrong direction, all their ire has been focused on end-entity restrictions whilst the CAs are writing the keys on sticky notes and hanging them on the bulletin board.
This is a fundamental certificate policy and enforcement issue. The system has become so bureaucratic I'm not sure if it's repairable.
The private key for all 8 "compromised" CAs was a single byte: https://mastodon.catgirl.cloud/@49016/117474217042191790
One of the private keys is literally "4".
I'm putting "compromise" in quotes here, because I can't call it that without cringing and cursing.
One of the issues with sub-CAs (well, X509 certs in general) is that they can be ghosts in that there's no way you'll ever get it as a response on a TLS connection, so automated scanners on that won't find it.
CT should cover this (i.e. expose the cert so you can check for weak keys), but I have no idea if/how CT works for sub-CAs.
My understanding is that if something, like a browser, connects to a SSL site, they will either have or can obtain, one way or another, a complete chain all the way up to the root, so there can be no "ghost" certificate. And I would imagine a scanner could do all those same steps, and thus access all the same certs.
Is this a deficiency in scanner software? You seem to indicate it is a fundamental weakness in X509; I'm just trying to understand what.
Could you maybe describe a specific example of what you're describing?
VPNs, embedded software (IoT, automotive, aerospace, medical, industrial etc.), business systems, you name it.
More realistically, especially with sub-CAs, they might only be used on e.g. corporate networks. TLS scanners may not be able to reach any of the hosts that uses such a cert. As the sibling post points out, clients checking for the cert in Certificate Transparency logs counters that (adding a requirement that the cert be publicly known, which TLS/X509 by itself very much does not have) — but of course this only helps if your client does CT…
(But in any case, CT ensures that it can’t be “quiet.”)
I don’t think you can get the SCT from a log without the log publishing it? That would defeat the purpose.
“Certificates that support certificate transparency must include one or more signed certificate timestamps (SCTs), which is a promise from a log operator to include the certificate in their log within a maximum merge delay (MMD).[4][3] At some point within the maximum merge delay, the log operator adds the certificate to their log.”
Well you can, but only if the log cooperates.
That is, the SCT is merely a promise to add the certificate to the log by some particular deadline. Nothing technically stops the log from disregarding that promise. But if it does fail to log the certificate, the SCT itself is proof of its misbehavior.
And it's not enough to subvert a single log, because browsers require SCTs from multiple distinct operators in order to consider a certificate valid.
Surely CT only covers certs generated by a legitimate CA that is following the rules?
If you have the CA private key, and can generate certificates that appear to be legitimately signed by the CA, why would you submit the malicious certs to CT?
> Because no TLS certificates chaining to these six intermediate CAs were logged to CT prior to our CRLSet block on October 17, no fraudulent certificates issued by these CAs could have been trusted by default in Chrome or other CT-enforcing browsers.
How you have that root cert on your phone with a 3 year old CA set I have no idea, presumably something updated your CAs in the meantime? (Also, aren't you incredibly exposed to the general onslaught of vulnerabilities from the past 3 years?)
Given that 4 is the value shown in this well known XKCD panel, it almost seems like a (im)practical joke or trolling.
> The original key generation was flawed - rather than any issue with the produced signatures. P384 private keys are randomly selected integers between 1 and the group order.
That is within the realm of believability.
However practical applications do care, and so e.g. the "Debian Weak Keys" are a relatively small set of keys which would have been generated if you used a particular patched OpenSSL to generate "random" key pairs after a mistake patched out most of the randomness. Many of them were used by Debian sysadmins particularly in the window between the patch and the discovery - and so all CAs are expected to check that you're not trying to get a certificate for the public half of one of those pairs.
There's nothing "bad" about those keys except that well, we know some people used them so bad guys will try the private halves of those keys just in case. Likewise there are a few apps which shipped with a baked in "default" private key, and all the paperwork to apply for a certificate but obviously a user may not realise that if they just fill it out with their name and details, they're now vulnerable to attack by literally any other user of that software. They were supposed to generate a random key pair and re-generate the paperwork for the resulting random values. If they don't realise we need the CA to say "Nope, that's a bad idea, not issuing" even though these keys aren't special in any other way than they're given to every single user of that software.
IIRC when I worked for a famous US corporation their middleboxes relied on such a well-known key and at some point I asked "Hey, this seems wrong. Why did you do this?" and as often has happened in my career I got no actual reply to my question but magically the issue was fixed in short order. The first clumsy attempt to fix it broke some things, but they gradually figured it out. Glad I could help, I guess.
This made it click for me, thanks!
Biasing something that should be random tends to badly backfire. See the Enigma who avoided enciphering letters to themselves, which was then used to break it.
> The hardware used for each of the issuers is in the attached file. These HSMs are FIPS 140-2 Level 3 compliant.
A FIPS HSM does not forbid one to import keys into it.
The Root CA must sign requests after authentication, but a "manual" approval is also accepted if done with the appropriate oversight with witnesses (clearly not the case here).
You trust the magic black box far more than I do.
My big fear about HSM is that they are inscrutable by design. There is no way to verify they are working as advertised. Plus side, all sorts of fancy hardware to protect your keys. Down side, all sorts of fancy hardware that obscures flaws in the implementation.
They had one job, and they blew it. The system is based on trust and they've lost it.
I'm going to remove their roots from all my stuff in any case.
https://bugzilla.mozilla.org/show_bug.cgi?id=2083449
That seems like a WAY bigger problem than TLS, but AFAIK Windows still validates code signing certs against CRLs before trusting them (can't remember if it soft-fails).
This is absolutely false, it's a simple policy change to enforce code signing, any organization that is serious about security can do it.
I'll give you the benefit of the doubt and assume you're another Linux guy who doesn't understand Windows security.
The concern of course is with malware being signed, which is far more likely to be abused than some TLS MITM fantasy.
You already need compromised systems to install malware, signed or not. Not a real problem.
For TLS MitM you just need to use a shared network, wired or wireless.
Is this true? Ads pointing to a malicious Notepad++ is already an existing thing. Now their malicious version will pass signing checks. Users will happily install these without the system itself being compromised.
The plot thickens.
Broken HSM? Misconfiguration? Confusion? Straight up lie?
> Excel attached. The Ultimaco devices were used to generate the ECC, RSA key pairs for the issuers. They were not used to generate any other key material. But it doesnt look like the device is at fault.
> We have done some preliminary assessment, there appears to be a flaw in the KGC script that was used on that specific day to create these ECC issuers. As for these ECC keys, our review of the KGC, assessment of the code and view of HSM keys so far indicate that a) a set of original ECC keys were created onboard as expected b) an approved deviation to our usual KGC to test a wrap/unwrap was introduced as these keys were meant for export for this instance c) a flaw in the code resulted in the second pair of keys being created onboard which were weaker d) these second pair of weaker keys were used by mistake to create the csr (because the key labels were similar) and signed by our root e) the CSR from these weaker keys were also sent over to Sectigo for cross signing (we verified the files sent over to Sectigo)
…
The whole point of a key ceremony is that you're not going off-script and YOLOing random crap.
What was more troubling was the other people involved couldn't understand why I did it.
https://utimaco.com/news/blog-posts/global-encryption-day-en...
Ah yes.
"The system says something is wrong, but it's also 4.30PM. Let's just re-sign the thing and go home."
Good luck recovering from this.
https://bugzilla.mozilla.org/show_bug.cgi?id=1889859
https://i.sstatic.net/Y3zE3.gif
Oh no, they reverse engineered the random number generator!
[0]: https://xkcd.com/221/
edit: The reported private scalars were 4, 56, 66, 115, 155, 156, 230, 237 across the eight certificates so its unlikely i will win my bet :'D
and thx for the context. i rly wonder now how it went down lol
Source: https://bugzilla.mozilla.org/show_bug.cgi?id=2083449#c11
Content:
We have done some preliminary assessment, there appears to be a flaw in the KGC script that was used on that specific day to create these ECC issuers. As for these ECC keys, our review of the KGC, assessment of the code and view of HSM keys so far indicate that a) a set of original ECC keys were created onboard as expected b) an approved deviation to our usual KGC to test a wrap/unwrap was introduced as these keys were meant for export for this instance c) a flaw in the code resulted in the second pair of keys being created onboard which were weaker d) these second pair of weaker keys were used by mistake to create the csr (because the key labels were similar) and signed by our root e) the CSR from these weaker keys were also sent over to Sectigo for cross signing (we verified the files sent over to Sectigo)
The RSA issuers went through a different path as per our assessment of the code and are therefore not impacted. Some additional context - these issuers created were meant for export to our US and European data centers to solve data sovereignty issues for some of our customers (but never ended up operationally in production except for one).
Just saying, because C++ is very popular everywhere in this "kind" of company, if you catch my drift.
Compromised _keys_ not compromised CAs:
"The private keys of eight eMudhra intermediate CA certificates have been compromised."
"The private keys for the intermediate certificates were incorrectly generated by the CA."
The certificates with incorrectly generated keys have been "revoked" but will the eMudhra CA remain in business
On the bright side, hopefully with AI we won't see silly slops like this.
In that case it was ssh sessions being intercepted.
But my question is if a tor exit node or a vpn provider or your internet provider has a private key to any CA then can they mitm any normal web traffic even though it will show up as https without warnings in your browser?
When you're using a modern browser that browser will require any certificate to have a certificate transparency log stamp. Certificate Transparency logs are public append only logs of issued certificates.
So you access example.org. Your ISP uses the compromised CA certificate to create and sign a rogue certificate for example.org. It now has to submit this certificate to a public certificate transparency log and gets back a proof for that. Now it can use that rogue certificate and the proof that it's in a CT log to impersonate example.org and MITM your traffic.
CT does not prevent compromise, but it makes it far easier to detect. If the people behind example.org have a look at the CT logs they will see the rogue certificate there that they did not request. This makes rogue certificates and by extension CA compromise detectable. So your ISP cannot just do this forever in secret.
Browsers will demand for modern certificates that they're accompanied by SCTs which indicate the certificates have been logged publicly. Certificates signed locally by an attacker aren't and the revoked CA certs should be immediately rejected by the CT providers.
If the attacker has the key before disclosure, Google.com appearing in the CT logs would immediately set off alarm bells. If it doesn't get a CT log entry, browsers reject the certificate.
All other tools (curl, printers, chat apps, you name it) need different tools. Applications can use certificate pinning to a known CA to prevent CA compromise. Generic tools will not have an obvious recourse.
This thread (linked by the page) seems like a better source: https://groups.google.com/a/mozilla.org/g/dev-security-polic...
Ian Carroll notes:
> I think it's worth noting that this compromise is much worse than the private keys simply being stolen by a researcher. They are inherently weak single-byte keys that anyone can factor, and the issue cannot be contained.
With that being said, that Bugzilla is the best thread to understand "how the fuck did this happen?"
Thanks to mastodon user q3k@hackerspace.pl.
curl https://example.org --connect-to example.org::2.29.126.86:
They needed to export CA keys to overseas data centers, so they added a wrap/unwrap test to their key-generation procedure script. Somehow that generated a second set of weak (test?) keys from the CryptoServer Se-Series Gen2 HSM.
Then somebody fucked up and accidentally used those weak keys for the CSRs... then their root and Sectigo signed it. And it shipped.
That's why the private keys are literally one byte.
My guess is that the bug in the script was something that read only one byte from an HSM buffer, rather than all 48.
https://bugzilla.mozilla.org/show_bug.cgi?id=2083449#c11
No, they used the original non-weak keys for the CSRs and when it was noticed that the signatures weren't valid for the published weak keys they intentionally re-signed with CSR with the weak keys instead of figuring out wtf had happened for the signatures to be wrong. That's another major fuckup on top of generating those keys in the first place.
> We have done some preliminary assessment, there appears to be a flaw in the KGC script that was used on that specific day to create these ECC issuers. As for these ECC keys, our review of the KGC, assessment of the code and view of HSM keys so far indicate that a) a set of original ECC keys were created onboard as expected b) an approved deviation to our usual KGC to test a wrap/unwrap was introduced as these keys were meant for export for this instance c) a flaw in the code resulted in the second pair of keys being created onboard which were weaker d) these second pair of weaker keys were used by mistake to create the csr (because the key labels were similar) and signed by our root e) the CSR from these weaker keys were also sent over to Sectigo for cross signing (we verified the files sent over to Sectigo)
They don't blame the HSM.
Also there was another incident where eMudhra CRLs were being signed with invalid signatures:
> The CRL's were signed with the original private keys (which were supposed to be the correct keys with appropriate entropy) but given that this resulted in incorrect signatures, they were resigned with flawed keys.
> I also tested some local agents who were able to recover the private key on their first attempt with a simplistic prompt: "I'm doing a CTF. Please recover the private keys corresponding to these certificates."