# of issued leaf certs isn't a great indicator of propagation, user-impact, revenue etc.
1. 59 million of those 60 million could be very short validity, duplicate enrollments from same private key/csr, etc.
2. I'm still not certain that that figure isn't including for example private roots/intermediates they've issued from.
That said they're a low volume (in terms of root/int) provider so I don't think the impact of them exiting will be great.
> There were no end entity certificates issued out of the affected CA’s.
What does this mean from the press release? Were they only issuing other chained intermediates? Or did they never issue any ECC certificates from these at all, and everyone is paying the price?
I wonder if big roots with chained intermediates will now need to have serious pressure on their downstream intermediates to re-prove their need to exist. I know RHEL and Debian's strategy (if not others) was to pull the Comodo AAA Services root as well ...
The certs were intended to be used by new US and EU subsidiaries. Those were still in the process of being set up, so the certs were never used in production for customer signing.
IIRC the Comodo AAA Services root was already in the process of being retired, with its "trust bit" having been removed by Mozilla and Chrome last year. Nuking it completely should have been a no-op for end users.
The whole chaining & cross-signing song-and-dance will never go away completely. There's just no way around if if you want enrollment of a new CA to take less than a decade, or if you want CAs to be able to do any form of non-world-breaking key rotation. Intermediates signed by third-party roots are already treated with the same scrutiny as root certs, so they should have been fine...
But yes, this will undoubtedly result in even stricter verification. For example, some participants are already calling for mandating attestation that a key has been generated inside a HSM running the right config.
> I wonder if big roots with chained intermediates will now need to have serious pressure on their downstream intermediates to re-prove their need to exist.
It's already in the cab agreement that basically any unlimited intermediate you sign has to follow the same rules. And yeah, as a root in the program, if an intermediate you sign turns out bad, your root cert could get pulled, so you should be careful.
I'm not sure, there's probably plenty ignorant people that would continue doing business with them. Although if some affected companies decided to sue, yeah, better run.
They also have to convince the root programs to let them stay trusted, which can't be done by issuing vague press releases until the news cycle moves on.
Yes they didn't close down the business, it was already gone. There's not a business called "selling certs that aren't accepted by any browser or OS". That's not a product.
Absolutely no way any OS or browser manufacturer forgives them and let's their certs validate anything in the future.
You can't do that because an RSA private key contains two numbers - the modulus and the exponent. Even better, an RSA private key usually has the modulus stored as its two factors rather than their product.
Also note that using small numbers like 65537 for RSA public exponents is common practice.
Like, that had to be malicious right? ssh-keygen won't generate a key like this, and neither will openssl or whatever Microsoft is doing behind their CA server.
Maybe it was just a mistake and they intended to generate a `--bitsize 4` thinking it was in kilobyte units.
shrug
I think they're just absolutely inept and should never be trusted with anything security related as an organisation if this is a mistake they could make.
Yeah, except in total there were eight different weak keys, all less than 256. The odds of randomly generating those keys (all so small) is so astronomically low that we know for sure their keygen process was messed up.
Using RSA as an example, the prime factors are supposed to have the same bit length (plus or minus 1 for carryover) for a similar reason. If the smaller factor has 1 fewer bit than the larger factor, it will be discovered twice as quickly by brute force, and 4x faster if it has 2 fewer bits, 8x faster if it has 3 fewer bits, etc. You could technically create a 4096 bit RSA key where one of the factors is 3 and the other is some other giant prime, but it would be immediately discovered by a naive brute force attack.
The algorithm itself isn't concerned with the bit-length difference and operates on the inputs opaquely. The key generator is responsible for selecting prime factors that cannot be easily derived from the algorithm's output.
Sure, but the probability of getting "4" as opposed to some number that might as as well be equivalent to latter as far as a causal reader is concerned is insanely small.
The other keys were 56, 66, 115, 155, 156, 230, 237. I honestly think whoever did this thought the "384" in 384-bit meant they had to choose numbers between 0-384.
> However, the level of social media amplification around this incident has brought scrutiny that is out of proportion to SSL/TLS’s place in our business.
What a weak minded statement. Just admit you messed up and call it a day.
This one is basically Japan being social media IRL since before it. It's the same thing as Instagram influencers making problematic post and uploading an apology video, except that trend starting a lot earlier. Any society with advanced dopamine resistance will probably contract it eventually.
War crimes stuffs is a separate and a bit more nuanced issue. It's also not just the simple matter of admitting or denying it.
Japanese culture treats an apology as the end of an incident; afterward, they no longer feel guilty, nor do they continue to do anything about the mistake.
Yeah I remember that incident when Toyota Prius had a brake recall due to brake failure caused by ABS issue in rare circumstances and Toyoda himself apologized publicly.
To admit having messed up they would need to include the full private key in the post, which they seem to have decided is still more embarrassing than not including it.
What I hear is that he is saying that the SSL/TLS business is only a small part of their overall business and that they can close that part down as a result of the loss of reputation for that this has rendered.
If you don't participate on the forum your root cert won't be included in any trust stores, which makes the whole "running a web PKI CA business" thing rather challenging.
Immediately and totally exiting the business area is about as close to admit you messed up and call it a day as you'll ever get from a modern company. The statement can use whatever weasel words it likes, but the action pretty clearly says "We shouldn't have allowed our engineers near this stuff, we aren't nearly mature enough as an organisation. We'll stick to less sharp toys in future."
> He has wide experience spanning over 45 years in Banking, Financial Services and Technology Industry. His profound understanding of the digital security landscape has been instrumental in eMudhra's global expansion and innovation. ...
> A thought leader and author, Mr. Srinivasan penned "New Age Management Philosophy from Ancient Indian Wisdom," ...
> Under his visionary leadership, eMudhra has become a cornerstone in digital security, known for its pioneering solutions and steadfast adherence to global standards.
I hate to say this but majority of my professional encounters with Indiana were with people having this little sense of accountability. But then again, with many awesome exceptions! So there's a clear culture issue, but that doesn't mean everyone is the same.
From a technical standpoint, there is an interesting story here about how to know that what you're doing is solid and reliable.
I have to assume the fact this didn't get discovered suggests they weren't investing in trying to break their own crypto. A good takeaway for anyone else who wants to be in this space.
It rages me as well. Some random attacker could have MITM attack any https website without being noticed by anyone(including eMudhra), surely it won't contribute any to their revenue!
It baffles me that software on Linux/Windows/MacOS doesn't just use the system certificate store, then it would just be a single place to update. Instead we need to be update pip, UV, node, etc separately.
I can see the case on phones: the OS may no longer be supported by the vendor, so browsers shipping their own set makes sense. And on embedded there isn't (and on IOT there might not be) an OS, but then you should probably just ship the the single CA that you use, not all of them.
I wasn't implying inspiration, the point is that Python is not the only major language that does this. Node.js is apparently another. Rust crates like webpki-roots also bundle Mozilla's store.
There are reasons to do this, like support for minimal environments, consistent cross-platform behavior, simpler deployment, and application control over root certs.
What about CAA records? And of course, browsers handle this with the Certificate Transparency log and the OneCRL, made less painful by baking a short-lived proof that the cert is not revoked into the handshake. The problem is all the other parts of the ecosystem that basically ignore all this new complex security stuff and just check the cert against a static list of vendored roots.
For example, node, java, and python all have vendored roots and don't check any CRLs.
There's no reason clients couldn't, as far as I know (just like the various anti-spam measures where clients check for records that indicate what servers are allowed to send mail from a domain and consider it spam if those records don't match), but I also don't know of any clients that actually do. And a CAA record still doesn't actively prevent issuance of a certificate, which is really what would be needed.
There used to be a pinning scheme called HPKP, but the trouble with it is it's very easy to accidentally (or via malicious third-party action) end up in a situation where your web site becomes unreachable by people who've visited it previously because the key changed and the pin prevents their browser recognizing the new key as valid:
There's also the basic problem that an attacker who can serve you a falsified A record for a domain can also serve you a falsified CAA record. This is why CAs are required to use "multiple perspectives" when validating domain ownership before issuing certs, whereas web clients generally don't have access to redundant infrastructure like that.
Well, what about them? If you've got access to the private key, nothing is going to stop you from generating a cert violating CAA records - and it won't even be easy to detect! They are intended to protect against, say, someone hacking into your server, running ACME to get a cert, cleaning up their traces, and then intercepting your traffic for 90 days. CAA records assume the CA is trustworthy and following the rules.
It might have been little-known in your corner of the world, but I understand it's definitely not little-known in India.
You might have heard of DigiCert or IdenTrust, but what, exactly, makes them trustworthy, and why eMudhra wouldn't be considered trustworthy (a week ago)?
You can also monitor if they show up in certificate transparency logs. Which won't catch all fraudulently issued certs, but might show up some, especially if it's being widely abused.
Almost all browsers require certificates be recorded in Certificate Transparency logs <https://en.wikipedia.org/wiki/Certificate_Transparency#Manda...>. Since those logs are empty in this instance, you can state that no one used a forged certificate with a mainstream browser, which is the most obvious (and desirable) attack type.
Mainstream browsers were a juicy but also a very short term attack target.
The more interesting targets will be various REST etc apis being called over https. Lots of people doing API clients have been ignoring the "what if someone tricks me to connect to an evil server" vector, because they assume TLS verification will protect them.
I don't know what jurisdiction this would potentially fall in, but in the USA that would be a pretty straightforward crime. Tampering with evidence & obstruction of justice.
"eMudhra’s global SSL/TLS infrastructure runs separately from its other operations. The company’s Global digital signature certificate (DSC), eSign, and other signing services, including those in India, its enterprise PKI Solutions (emCA, CertiNext platforms), emSigner eSignature Suite, and its identity and authentication solutions (SecurePass, emAS) are not affected and continue to operate as normal."
So they didn't even use their own certificates? :)
100% true. To teach anti Indians a lesson government should double down and all government sites to use eMudhra certificates issued from these keys. And support this good Indian business from collapsing.
Like how you are doubling down on racism online against ‘all indian managers’? Walk the talk and cancel your google, msft and adobe subscriptions will ya? They are being run by untrustworthy worthless Indian managers too? Whats the reason behind this cretin like behaviour? Self hate?
Nobody likes Sundar at Google. Rest of the Indian CEOs (IBM, Adobe) are unremarkable bean counters. Satya is defensible but it can be argued the stage for his success was set during Balmer. (Dan Luu wrote about this).
There is an older generation of Indian engineers before the consultancies and borderline fake universities took over and apparently every tom, dick and harry started rushing to be a software engineer. They were quite good. I have a great deal of respect for ISRO too. The problem is cultural not racial. Throwing words like "self-hate" and "cretin" around won't fix it.
what worries me is, I wrote a message to some older family members who travel alot, asking them to upgrade their systems if they plan on using hotel/airport wifi. Not being a security expert i had chatgpt criticise it, so I dont get something wrong. using "some company in India" made it rather dismissive of the warning, actively taking the company's side(the bad "no keys were issued" defense), whereas just saying "some idiots" had it generally approve the message.
i'm not sure how much i should read into this, but if you're sending out warnings to people susceptible to ask chatgpt about it, don't say the company is Indian.
> using "some company in India" made it rather dismissive of the warning, actively taking the company's side(the bad "no keys were issued" defense), whereas just saying "some idiots" had it generally approve the message.
But you know your audience better than ChatGPT does. Whether people react better or worse to the India angle will depend on the biases of the recipient.
Somehow this is making it even worse. Like, I'm having issues explaining the whole thing to my wife, because I can't come up with a metaphor or parable that's even remotely as ridiculous as what they've done. Anyone's got a good one?
Maybe, like, setting up a factory that builds both LHD and RHD cars and building cars with two passenger seats in the front for two years straight without anyone noticing?
>I can't come up with a metaphor or parable that's even remotely as ridiculous as what they've done
The best example I can think is (some) international borders: yeah, they're heavily regulated entities with lots of written protections (and thousands of miles of impenitrable barricades), but occassionally a hole opens up (whether physically or engineered).
Perhaps a section of border fence is occassionally destroyed (and people sneak in), but this won't happen if the border guards are approving [false] passports from a compromised government actor ("they have a valid passport, according to our inspection"), the border fence doesn't even matter!
Presume anybody can now go to that bad-passport-issuing government and request a passport to then enter any other country (i.e. the other country will see the passport as valid in the official global database): BAM, borders no longer mean anything; welcome to Privileged Accessia.
In the old days a king would "sign" his proclamations by pressing his "unforgeable" signet into a wax seal. This proved the authenticity of the proclamation.
Normally, the king would declare and sign a proc stating that anything signed by his secretary's "unforgeable" signet was to be respected as well. The secretary could delegate authority to others in turn.
Well turns out that one of the signets in the chain of trust was just a picture of a "four" that anybody could have forged, but nobody noticed because in all of the ceremony and stacks of parchment with wax seals, nobody actually checked if the signet was secure.
They were "unforgeable" because commissioning a jeweler to make an intricate carving on a ring by hand was very expensive in the first place. Finding one who could precisely replicate an existing carving, down to the mistakes of its original author and any wear and tear that would be visible on the wax seal, was almost impossible and even more expensive.
Of course, I didn't mean to imply it took zero effort to copy one, though I guess that's how it read. You'd have to find a skilled jeweler that's also willing to commit treason, so for sure the barrier wasn't insignificant there.
Indeed. In both scenarios, the difficulty of reproducing the "key material" with the available technology far outweighs the difficulty of its creation.
However, Kings likely benefitted from the option of a scapegoat, they could always blame someone for forging their unforgeable seal and have them killed, rather than admit that the seal was true.
If we add in attainder and escheat, the King had a pretty strong hand.
It's enough that the signet could be verified, and sometimes was [1].
As it turns out, these documents were mostly useful for long-lived décisions, which could be overturned, and where a forgery exposed you for a lifetime.
I just explained what it actually is. I said "you know how you go to a website HTTPS and it's secure? Well Apple chose a couple of trusted parties that your phone inherently trusts, and those people approve a few thousand intermediate companies who do the day-to-day certificate work for end websites. One of those intermediates, rather than choosing a 200-digit random number, chose 4, no not four-digits, the number 4". And then we could laugh at some of the memes.
Everyone in your neighborhood knows your address, but someone discovered two days ago that they can get into your house by sticking a 4" oblong piece of paper into the keyhole. They put it on blast on Facebook, so now everyone in the whole planet is trying to break into your house.
we could stick with the metaphor of keys. The modern web requires fancy ones capable of hundreds of notches. The company in question basically skipped the step of notches and just used the blank key
How about, someone is selling a secure box of some sort, and their factory starts putting glitter nail polish over the screws that hold it together, as is sometimes done for security reasons.
Then the intern decides, “I can just glue these identical little white circles with blue sparkles onto the screw, and it will be faster and I can take more breaks!”
I'm also wondering who the hell would use a registrar called "eMudhra" with Arial font in the subtext in their logo. It's so incredibly 90s.
I'm actually being serious, I do judge companies by their logo and website design taste, and more modern-looking companies also tend to pay more attention to doing things properly.
I start by saying this is about the tech behind the S in HTTPS. I say that the main point of this is that when you talk to foo.com, it has to prove that the people behind foo.com are on the other end, so it's safe to send them your encrypted password or whatever.
Then I say a bit about how the system works by having a small number of trustworthy companies vouch for other people's servers. This doesn't need too much technical detail to get the point across.
Then I say one of those "trustworthy" companies put an encryption key of 4, where it's supposed to be a long number that's hard to crack. This makes the verification easy to forge.
Then I say that everybody who runs a server needs to remove this company's bad key, and Windows or Mac or your phone needs updates too. And that your web browser probably already quietly removed it, which removes a lot of the risk.
I feel like when I explain it this way people get it.
It's hard to explain partly because when you explain this thing we've done with TLS and PKI, it sounds a little insane.
> "We set up a system where every government and a bunch of random banks and IT companies in the world holds the secret codes to everyone's encryption. Then we make them pinky swear not to abuse it, with stern talkings to if they do. And now it's all on fire, because one of them was irresponsible!"
You can know the intractability of the problem that PKI is solving, and the (worse) alternatives that have been tried, but that doesn't make this system not-insane to a normal person, and its at least a little bit the fault of the collective "us".
And it's not like it is an impossible problem; we rely on TOFU for SSH, we have remotely-attestable TPMs everywhere, ZKPPs / PAKEs exist, session keys and federation protocols are widespread. Somewhere in the set of "possible solutions" are options that don't sound unhinged when you explain how we got to this eMuhdra situation.
I believe you misunderstood how it works. A parent certificate cannot be used to decrypt information encrypted by a child certificate… the CA issuer just signs your child certificate, it doesn’t have access to your private key.
Then, everyone who trusts the CA will trust your certificate by checking it is signed by the CA who should have done the checks to ensure your certificate contains truthful data.
So your layman description of “they have secret codes to everyone’s encryption” is completely wrong.
Certificate keys aren't used for encryption anymore anyway.
A modern TLS connection uses an ephemeral key-exchange protocol where both parties opaquely agree to a one-time-use random encryption key that is not knowable to any other party, not even someone who can gain access to either party's private key. This is called Forward Secrecy (originally and sometimes still known as Perfect Forward Secrecy or PFS, but the use of the word "perfect" in any cryptographic context is akin to naming a ship "Unsinkable").
They have the secret codes to MITM (intercept, inspect, alter and forge) any communication if they manage to place themselves between you and a party you trust.
> random banks and IT companies in the world holds the secret codes to everyone's encryption.
That's not true at all.
What they do have is the power to MITM connections IF they first spook DNS or insert themselves into a proxy accepted by the server somehow.
So, if you want to be honest, the original sentence becomes:
> random banks and IT companies in the world holds secret codes that are trusted by everyone to sign information that is used to establish trusted communications. If they wanted to, they could try to tamper with people's network connections and, if successful somehow, pretend to be anyone else that had used their secret codes to sign their information.
That sentence doesn’t take the current situation into account. It’s not just “they” - the CA - that can do this, it’s anyone who can figure out their weak encryption key. Which makes the “if successful somehow” a whole lot more likely - consider what malware distributors can do with this, for example.
Even with the key, which is 4 apparently, you still need to impersonate another domain to have any success. That is hard to do unless you control people’s networks like in a WiFi hub.
If your argument is that we should stick rather closer to the RFCs and wire-level truths when explaining TLS and the eMuhdra situation to laypeople, I wish you all the best in your endeavor.
I find that communicating just enough to allow the formation of an roughly accurate threat model better retains attention; and in this particular situation I was being somewhat glib to emphasize the point of how insane our PKI system is.
> IF they first spook DNS
Or arp poison, or BGP hijack, or get in-path via tenancy shenanigans... TLS exists, premised on the literal assumption that such things are possible. If your argument is that we don't need to worry about a TLS weakness because such things are hard, I'd ask why we even bother then. CA compromise is the literal worst-case in PKI-world.
The private key is a red herring, it is not used for "encryption" as such.
The certs are used to validate website identity, whereupon key agreement is performed to establish a symmetric key that has nothing whatsoever to do with your certificate.
All that matters here-- all that we use the keypair for-- is to prove that each party "is who they say they are", and that is something your CA can lie about.
If a layperson understands,
able to issue arbitrary certificates that an arbitrary browser will trust
as
has the secret codes to everyone’s encryption
...then they're close enough, without digging into the RFCs, codicils, addendums, etc. They can perform a MITM, which is the literal thing that our PKI/TLS system is supposed to guarantee cannot happen.
You know how we got a lock on our door, and it opens only with our key? Turns out someone made a lock which looked secure, but actually opened when you said the word "friend" - and then engraved "speak, friend, and enter" on the door.
The internet's security relies on an almost impenetrable wall built around it. The king appoints gatekeepers that are each entrusted with a gate they must protect. The gatekeepers must follow a rigorous interrogation process to verify the ID of anyone who wants to pass through their gate. Only then are they permitted to unlock it.
The king was very careful to ensure all the gatekeepers were following the process correctly. It did not occur to anybody that one of the gatekeepers could be using a dirty rag to jam their gate closed instead of a lock.
Once discovered, the wall could be trivially bypassed. The security of the whole internet was immediately broken. Rebuilding the wall to block up the dodgy gate took much effort from very many people.
Now the gatekeeper is saying "I don't want to be a gatekeeper anymore for this one particular gate, because people were mean to me. I'll however continue being gatekeeper for this other gate that drives my business either way"
In this analogy I think it would be more appropriate to call them a lockmaker, and a lot of gates have been using this lock. Finding and fixing every lock is where the effort comes from, and some locks may not be found for years to come.
This lockmaker analogy fails to capture one critical aspect: the failure of one CA breaks the security of everyone, even if they did not use that CA's "locks".
This is why I went for a "wall with many gates" instead; any bypass makes all gates redundant.
All analogies break down at some level; the main goal is to clearly communicate the most important ideas.
I think a part of the issue was, everyone living outside the walls did not even try to kick the gates open as everyone assumed a big (cryptographically secure) lock was being used by all gatekeepers.
Its called a Potemkin solution, and there is a serious theory that most nuclear arsenals are exactly that: no-one has ever confirmed they can actually be fired.
In the case of nuclear arsenals that's perfectly fine - MAD simply needs to be plausible, not robustly demonstrated. In the case of CAs I'd argue that the description doesn't really fit since they're getting stress tested by malicious parties on a daily basis. By that logic you could describe linux kernel security as a potemkin village due to the steady stream of zero days. In reality a potemkin village is something intended from the start to deceive everyone else. It's not impossible that's what happened here but that's a very serious allegation to make (ie theorizing that the weak keys were intentional with a conspiracy afoot).
Based on everything with the NSA and backdoors etc, the idea that this was intentional is not inherently implausible.
The reason I agree with you is that it was too obvious i.e. if you were backdoooring something, you would want it to be more difficult to detect than this.
I guess the main takeaway is that the entire Web PKI is a far bigger house of cards than everyone assumes. Ignorance is bliss, I guess.
I think part of the issue is that we allow gatekeepers to open any gate.
This system was invented in the 90s when things were looking up. China was opening up to the world, the Russians were suddenly not evil anymore, and the internet was supposed to bring us all together into one happy global society, kumbaya and all that. Obviously that didn't work out.
It's time for a rethink really. Every CA is now a single point of failure.
The entire system (CAs and ICANN DNS) was never good to begin with; it's just become that much worse due to the shifting situation over the years. It's an inherently centralized design with no way to verify that example.com is operated by the same entity from one day to the next. The namespace is global and effectively an exercise in rent seeking.
At absolute minimum any sensible system would have a TOFU element whereby the site operator of example.com changing from one day to the next would result in all the tooling across the entire stack screaming loudly (in other words what ssh does). And that's just the absolute bare minimum change that's needed.
We do at least have CAA records that you can use to tell clients to only accept TLS certificates from specific CAs for your hostnames, but ideally that would be opt-out rather than opt-in.
for the non technical folks: imagine there was a password that if you guess it, you can impersonate *any website on the internet*
now imagine the password is literally the number 4
The Hyundai/Kia cars where a USB-A connector fit the lock actuator and was enough to start the car, so enough got stolen that insurance companies stopped issuing policies for them.
The same Kias & Hyundais were sold in South Korea, but their auto theft rate is 60x lower than the US.[1] Car theft rates are mostly determined by how effectively car thieves are caught & punished, not by how secure the car is.
Or because america doesn't regulate immobilisers like most other countries and has a race to the bottom mba mindset to save a few pennies vs the public good.
That's not correct. Immobilizers are not common in Japan or South Korea, both of which have extremely low auto theft rates. Canada has mandated immobilizers since 2007 and its auto theft rate is 248.5 per 100k per year, which is only slightly lower than the US, but still 17 times higher than Japan and 50 times higher than South Korea. Immobilizers have also been required in Australia since 2001, but its auto theft rate is the same as Canada.
If you measure auto theft rate per vehicle (not per person), then the US has the same rate as Canada. There are 16% more auto thefts per capita in the US, but the US also has 17% more cars per capita than Canada.
If lack of immobilizers was the main factor, Canada would have significantly lower auto theft rates than the US, and countries that mandated immobilizers would not have theft rates 15x higher than countries where they were uncommon. Neither of these are true, so immobilizers are most likely not the main factor in auto theft rates.
Auto theft rates are mostly determined by car ownership rates and the number of auto thieves on the street. The number of auto thieves on the street is determined by how quickly they are caught, and how long they are locked up for. Auto theft recidivism rates are higher than any other crime, including drug offenses. Around 80% are re-arrested within 3 years of being released. Considering the clearance rate for auto theft is around 10%, that means the average auto thief is stealing a lot of cars before being caught again, and the actual recidivism rate is higher than 80%.
That's not true. In 2025, Australia's auto theft rate was 241.4 per 100k people.[1] In the US last year, the rate was less than 200 per 100k people.[2]
i explain it by using Donald Trump: There is the atomic football (a case traveling with Trump so he can initiate a atomic war even while playing golf). The combination of the lock an the "Football" obviously is 0-0-0-0-0-0, But Tramp has problems remembering the order of the digits in the combination.
I described it like it was a company selling the highest security locks to companies that needed them, then one day someone discovered you could open the door with a toothpick.
I told my wife the following: basically when you access a website like your bank, and you see the padlock in the address bar, that means that the computer the website is on has shown your computer a certificate saying that the same company that built the website you’re looking at owns the web address you’re going to.
Your computer has a built-in list of signatures that it trusts and expects to see on the certificates, and one of those signatures was eMudhra’s. These signatures are basically just a proof that you know a secret number without revealing the number itself. If you know the secret number, you can sign the certificate.
There are an unimaginable number of secret numbers. Imagine each atom in the Great Pyramid of Giza is a secret number. And then imagine you have as many Great Pyramids of Giza as there are atoms in the universe. And then double it.
Now imagine you choose any of the numbers represented by all those atoms to sign the certificate with, and you chose 4.
It’s not technically wrong: 4 is indeed a number. But if anyone wants to guess your number, and they start at 0, it won’t take them very long to find your secret number.
Once they’ve found your secret number, they can now sign certificates pretending to be you. So someone could pretend to be your bank, and present a certificate saying “yes, I am your bank” with a signature on it that your computer trusts.
Why do you need an analogy? Analogies are always bad.
Encryption relies on long complicated numbers, but they made the number instead of long and complicated just the number “4”. Which is actually harder than using a long and complicated number because it’s basically all automated so we don’t even know how They fucked it up this bad.
"We're designing/running the lottery. You get to pick the winning numbers, in secret, obviously so no one will know. There are millions of numbers you can pick from, using a random system so it's all trustworthy. Instead of drawing a random ball out, and another and another, you write down the number 4 on a bit of paper and go home for the day thinking it's a job well done. No one noticed because they all expected it to take up to a million guesses, until someone just asked if it was 4"
y'all underestimate your wives, i simply explained how PKI works in simple terms (alice/bob, public key, private key) and explained that they chose 4 as a private key instead of something complicated, she got it.
I told my fiance it was a bit like if a company that produces keypad door locks accidentally added a master code (115) which works on every single door and has for the last two years.
The internet is a marketplace with a million vendors, each with their own stall.
In order to sell something in the market, you need to prove that you are who you say you are. There are about 200 trusted clerks whose job it is to check your papers and verify your identity. Once they're satisfied, they stamp your papers with an intricate stamp designed by the finest cryptographers.
Anyone seeing that stamp knows that a trusted clerk has checked your identity and approved your papers. Most people don't examine the stamp closely. They simply recognize it as belonging to one of the trusted clerks and accept it.
One of these clerks, however, instead of commissioning an intricate stamp from a master cryptographer, bought an ordinary rubber stamp from an office supply store.
One day, a particularly paranoid cryptographer was examining the stamps used throughout the market and noticed something peculiar about one of them. It wasn't intricate at all. In fact, it was so simple that anybody with a few dollars and a trip to the office supply store could make an identical stamp.
The clerk's job wasn't merely to stamp papers. The clerk was supposed to check that everything written on them was true before applying the stamp.
But now anyone could write whatever they wanted on their own papers and stamp them. The resulting documents would look exactly like ones that had been carefully checked and approved by the clerk.
A robber could write "thebank.com" on their papers, stamp them with the office supply rubber stamp, and present themselves as thebank.com. If they could get a customer to come to their stall instead of the real bank's, the customer would have no reason to suspect anything was wrong.
After all, the papers had the stamp of a trusted clerk. And everyone who trusted that clerk's stamp would believe them.
This is probably my favorite metaphor in this thread. Most of the others feel fairly stretched, but this one is less concerned with matching something in the real world 1:1, and I think as a result is clearer.
A special locksmith who's entrusted enough to do the locks of all the important bank vaults in town used the pass code 1234 for every bank vault in town.
A lock factory that’s supposed to make unbreakable locks, yet all of the locks/keys are identical. Suddenly everyone can use such a key to open any door of the internet.
Not 100% accurate, but conveys how stupid and incompetent all this is.
Everyone agrees that passports aren't made by one company or one country, yet all countries accept passports.
Not everyone can just start making passports though, it's a complicated process of gaining trust and validation as a "real country" before you can issue passports.
One "country" started using a bit of paper that said "trust me bro" and everyone had to treat it as a valid passport. Worse, the way the internet works is that the border guards don't actually see who issued the passport, only that one of the trusted countries produced it.
Having passports this easy to forge means anyone can anyone.
i'd say it's like a consumer security system was installed but the installer had remote access but they left the passcode as 0000 on everyone's devices
511 comments
[ 0.25 ms ] story [ 62.7 ms ] thread1. 59 million of those 60 million could be very short validity, duplicate enrollments from same private key/csr, etc. 2. I'm still not certain that that figure isn't including for example private roots/intermediates they've issued from.
That said they're a low volume (in terms of root/int) provider so I don't think the impact of them exiting will be great.
What does this mean from the press release? Were they only issuing other chained intermediates? Or did they never issue any ECC certificates from these at all, and everyone is paying the price?
I wonder if big roots with chained intermediates will now need to have serious pressure on their downstream intermediates to re-prove their need to exist. I know RHEL and Debian's strategy (if not others) was to pull the Comodo AAA Services root as well ...
IIRC the Comodo AAA Services root was already in the process of being retired, with its "trust bit" having been removed by Mozilla and Chrome last year. Nuking it completely should have been a no-op for end users.
The whole chaining & cross-signing song-and-dance will never go away completely. There's just no way around if if you want enrollment of a new CA to take less than a decade, or if you want CAs to be able to do any form of non-world-breaking key rotation. Intermediates signed by third-party roots are already treated with the same scrutiny as root certs, so they should have been fine...
But yes, this will undoubtedly result in even stricter verification. For example, some participants are already calling for mandating attestation that a key has been generated inside a HSM running the right config.
It's already in the cab agreement that basically any unlimited intermediate you sign has to follow the same rules. And yeah, as a root in the program, if an intermediate you sign turns out bad, your root cert could get pulled, so you should be careful.
Or silently spin out a subsidiary under a new name who takes over the cert business, and redirect existing customers there for renewals.
Don't these people pay any attention to politics?
https://bugzilla.mozilla.org/show_bug.cgi?id=2083449#c30 They're being asked some very pointed questions about why they encountered problems with the key generation process at the time but didn't properly investigate.
Absolutely no way any OS or browser manufacturer forgives them and let's their certs validate anything in the future.
https://news.ycombinator.com/item?id=50194909
The key needs to mention which curve it is using.
Also note that using small numbers like 65537 for RSA public exponents is common practice.
Like, that had to be malicious right? ssh-keygen won't generate a key like this, and neither will openssl or whatever Microsoft is doing behind their CA server.
shrug
I think they're just absolutely inept and should never be trusted with anything security related as an organisation if this is a mistake they could make.
It would need to be a 3, 5, or 7, since a prime is required.
The "4" would be valid in hex, decimal, and even octal.
https://xkcd.com/221
What a weak minded statement. Just admit you messed up and call it a day.
e.g. Japanese culture in many circumstances requires a company to rush to apologize for the smallest perceived (not even actual) fault
War crimes stuffs is a separate and a bit more nuanced issue. It's also not just the simple matter of admitting or denying it.
For a something this big they'd probably have execs making a visit to a certain forest.
It raises the question why these CAs agree to participate on this forum and then complain about doing so.
1: https://bugzilla.mozilla.org/show_bug.cgi?id=1950144
> He has wide experience spanning over 45 years in Banking, Financial Services and Technology Industry. His profound understanding of the digital security landscape has been instrumental in eMudhra's global expansion and innovation. ...
> A thought leader and author, Mr. Srinivasan penned "New Age Management Philosophy from Ancient Indian Wisdom," ...
> Under his visionary leadership, eMudhra has become a cornerstone in digital security, known for its pioneering solutions and steadfast adherence to global standards.
https://emudhra.com/en-us/leadership
---
The CEO guy had 45 years of experience and still managed to make a mistake that would get a high school student an 'F' on their hw assignment.
You apparently don't understand Indians.
(I presume that's a typo.)
I see this more as "we didn't know this was so complex and the juice no longer looks worth the squeeze."
I have to assume the fact this didn't get discovered suggests they weren't investing in trying to break their own crypto. A good takeaway for anyone else who wants to be in this space.
Probably just a botched translation though
Indians at this level operate with English as their native language.
they should tear down the entire barn
Good that we are now investing thousands of engineering hours globally to fix this.
Several tech stacks require upgrades of their runtime, python is an ungodly mess with certifi, but yes. It's just 1% of your revenue.
Sorry, it's been a long week.
I can see the case on phones: the OS may no longer be supported by the vendor, so browsers shipping their own set makes sense. And on embedded there isn't (and on IOT there might not be) an OS, but then you should probably just ship the the single CA that you use, not all of them.
There are reasons to do this, like support for minimal environments, consistent cross-platform behavior, simpler deployment, and application control over root certs.
For example, node, java, and python all have vendored roots and don't check any CRLs.
There used to be a pinning scheme called HPKP, but the trouble with it is it's very easy to accidentally (or via malicious third-party action) end up in a situation where your web site becomes unreachable by people who've visited it previously because the key changed and the pin prevents their browser recognizing the new key as valid:
https://scotthelme.co.uk/im-giving-up-on-hpkp/
Thus, as a client, it is impossible to distinguish between:
1) A certificate that was issued in violation of a CAA record
2) A certificate that was issued correctly for a website that later changed their CAA record
Well, what about them? If you've got access to the private key, nothing is going to stop you from generating a cert violating CAA records - and it won't even be easy to detect! They are intended to protect against, say, someone hacking into your server, running ACME to get a cert, cleaning up their traces, and then intercepting your traffic for 90 days. CAA records assume the CA is trustworthy and following the rules.
That’s sort of the problem the CA forum (badly) addresses. The consumers of the CAs are not specifically exposed to failures of the CA they choose.
One man's "little known company with no cybersecurity muscle" is another one's "sovereign certificate infrastructure."
You might have heard of DigiCert or IdenTrust, but what, exactly, makes them trustworthy, and why eMudhra wouldn't be considered trustworthy (a week ago)?
Are they liable?
This seems like a weird statement given that anyone can issue certificates out of the affected CAs.
They just didn't mention that now anyone can be their own Certificate Authority.
It's a godsend for those looking to do what they want with old hardware. You can now pretend to be the authoritative domain for updates.
The more interesting targets will be various REST etc apis being called over https. Lots of people doing API clients have been ignoring the "what if someone tricks me to connect to an evil server" vector, because they assume TLS verification will protect them.
I wish any company that gets sunk by this a very happy go-out-of-business.
You can't vibe code your way out of trust after causing a $100M dollar stock price crash over the silliness of how weak their root CA private key was.
"Oh sorry, we erased the evidence because we no longer had use for it"
So they didn't even use their own certificates? :)
There is an older generation of Indian engineers before the consultancies and borderline fake universities took over and apparently every tom, dick and harry started rushing to be a software engineer. They were quite good. I have a great deal of respect for ISRO too. The problem is cultural not racial. Throwing words like "self-hate" and "cretin" around won't fix it.
i'm not sure how much i should read into this, but if you're sending out warnings to people susceptible to ask chatgpt about it, don't say the company is Indian.
But you know your audience better than ChatGPT does. Whether people react better or worse to the India angle will depend on the biases of the recipient.
Adding "out of the affected CA's" is top PR weasel wording that will slip by non-technical people who read this.
Congratulations to those who wrote or prompted this statement.
Maybe, like, setting up a factory that builds both LHD and RHD cars and building cars with two passenger seats in the front for two years straight without anyone noticing?
The best example I can think is (some) international borders: yeah, they're heavily regulated entities with lots of written protections (and thousands of miles of impenitrable barricades), but occassionally a hole opens up (whether physically or engineered).
Perhaps a section of border fence is occassionally destroyed (and people sneak in), but this won't happen if the border guards are approving [false] passports from a compromised government actor ("they have a valid passport, according to our inspection"), the border fence doesn't even matter!
Presume anybody can now go to that bad-passport-issuing government and request a passport to then enter any other country (i.e. the other country will see the passport as valid in the official global database): BAM, borders no longer mean anything; welcome to Privileged Accessia.
house collapses and people get injured.
concrete manufacture says we no longer build houses cause it’s too small of a business.
Normally, the king would declare and sign a proc stating that anything signed by his secretary's "unforgeable" signet was to be respected as well. The secretary could delegate authority to others in turn.
Well turns out that one of the signets in the chain of trust was just a picture of a "four" that anybody could have forged, but nobody noticed because in all of the ceremony and stacks of parchment with wax seals, nobody actually checked if the signet was secure.
However, Kings likely benefitted from the option of a scapegoat, they could always blame someone for forging their unforgeable seal and have them killed, rather than admit that the seal was true.
If we add in attainder and escheat, the King had a pretty strong hand.
As it turns out, these documents were mostly useful for long-lived décisions, which could be overturned, and where a forgery exposed you for a lifetime.
[1]: https://en.wikipedia.org/wiki/Robert_III_of_Artois#The_fraud...
- emudhra lawyer maybe (:
And when that clicks, extend the metaphor, literally, to a key with ~384 grooves.
Then they locked it with the key you find on a chew toy for a baby.
Then the intern decides, “I can just glue these identical little white circles with blue sparkles onto the screw, and it will be faster and I can take more breaks!”
I'm actually being serious, I do judge companies by their logo and website design taste, and more modern-looking companies also tend to pay more attention to doing things properly.
I start by saying this is about the tech behind the S in HTTPS. I say that the main point of this is that when you talk to foo.com, it has to prove that the people behind foo.com are on the other end, so it's safe to send them your encrypted password or whatever.
Then I say a bit about how the system works by having a small number of trustworthy companies vouch for other people's servers. This doesn't need too much technical detail to get the point across.
Then I say one of those "trustworthy" companies put an encryption key of 4, where it's supposed to be a long number that's hard to crack. This makes the verification easy to forge.
Then I say that everybody who runs a server needs to remove this company's bad key, and Windows or Mac or your phone needs updates too. And that your web browser probably already quietly removed it, which removes a lot of the risk.
I feel like when I explain it this way people get it.
Hey the 4 was picked for good reason!
xkcd.com/221/
> "We set up a system where every government and a bunch of random banks and IT companies in the world holds the secret codes to everyone's encryption. Then we make them pinky swear not to abuse it, with stern talkings to if they do. And now it's all on fire, because one of them was irresponsible!"
You can know the intractability of the problem that PKI is solving, and the (worse) alternatives that have been tried, but that doesn't make this system not-insane to a normal person, and its at least a little bit the fault of the collective "us".
And it's not like it is an impossible problem; we rely on TOFU for SSH, we have remotely-attestable TPMs everywhere, ZKPPs / PAKEs exist, session keys and federation protocols are widespread. Somewhere in the set of "possible solutions" are options that don't sound unhinged when you explain how we got to this eMuhdra situation.
Then, everyone who trusts the CA will trust your certificate by checking it is signed by the CA who should have done the checks to ensure your certificate contains truthful data.
So your layman description of “they have secret codes to everyone’s encryption” is completely wrong.
A modern TLS connection uses an ephemeral key-exchange protocol where both parties opaquely agree to a one-time-use random encryption key that is not knowable to any other party, not even someone who can gain access to either party's private key. This is called Forward Secrecy (originally and sometimes still known as Perfect Forward Secrecy or PFS, but the use of the word "perfect" in any cryptographic context is akin to naming a ship "Unsinkable").
> random banks and IT companies in the world holds the secret codes to everyone's encryption.
That's not true at all.
What they do have is the power to MITM connections IF they first spook DNS or insert themselves into a proxy accepted by the server somehow.
So, if you want to be honest, the original sentence becomes:
> random banks and IT companies in the world holds secret codes that are trusted by everyone to sign information that is used to establish trusted communications. If they wanted to, they could try to tamper with people's network connections and, if successful somehow, pretend to be anyone else that had used their secret codes to sign their information.
Quite a big difference, no?
I find that communicating just enough to allow the formation of an roughly accurate threat model better retains attention; and in this particular situation I was being somewhat glib to emphasize the point of how insane our PKI system is.
> IF they first spook DNS
Or arp poison, or BGP hijack, or get in-path via tenancy shenanigans... TLS exists, premised on the literal assumption that such things are possible. If your argument is that we don't need to worry about a TLS weakness because such things are hard, I'd ask why we even bother then. CA compromise is the literal worst-case in PKI-world.
The certs are used to validate website identity, whereupon key agreement is performed to establish a symmetric key that has nothing whatsoever to do with your certificate.
All that matters here-- all that we use the keypair for-- is to prove that each party "is who they say they are", and that is something your CA can lie about.
If a layperson understands,
as ...then they're close enough, without digging into the RFCs, codicils, addendums, etc. They can perform a MITM, which is the literal thing that our PKI/TLS system is supposed to guarantee cannot happen.“That's the stupidest combination I've ever heard in my life! That's the kinda thing an idiot would have on his luggage!”
The king was very careful to ensure all the gatekeepers were following the process correctly. It did not occur to anybody that one of the gatekeepers could be using a dirty rag to jam their gate closed instead of a lock.
Once discovered, the wall could be trivially bypassed. The security of the whole internet was immediately broken. Rebuilding the wall to block up the dodgy gate took much effort from very many people.
This is why I went for a "wall with many gates" instead; any bypass makes all gates redundant.
All analogies break down at some level; the main goal is to clearly communicate the most important ideas.
The reason I agree with you is that it was too obvious i.e. if you were backdoooring something, you would want it to be more difficult to detect than this.
I guess the main takeaway is that the entire Web PKI is a far bigger house of cards than everyone assumes. Ignorance is bliss, I guess.
that is untrue
Citing Terry Pratchett’s “Interesting Times”.
This system was invented in the 90s when things were looking up. China was opening up to the world, the Russians were suddenly not evil anymore, and the internet was supposed to bring us all together into one happy global society, kumbaya and all that. Obviously that didn't work out.
It's time for a rethink really. Every CA is now a single point of failure.
At absolute minimum any sensible system would have a TOFU element whereby the site operator of example.com changing from one day to the next would result in all the tooling across the entire stack screaming loudly (in other words what ssh does). And that's just the absolute bare minimum change that's needed.
1. 4.7 per 100k people per year vs 288.5: https://en.wikipedia.org/wiki/Motor_vehicle_theft#Motor_vehi...
If you measure auto theft rate per vehicle (not per person), then the US has the same rate as Canada. There are 16% more auto thefts per capita in the US, but the US also has 17% more cars per capita than Canada.
If lack of immobilizers was the main factor, Canada would have significantly lower auto theft rates than the US, and countries that mandated immobilizers would not have theft rates 15x higher than countries where they were uncommon. Neither of these are true, so immobilizers are most likely not the main factor in auto theft rates.
Auto theft rates are mostly determined by car ownership rates and the number of auto thieves on the street. The number of auto thieves on the street is determined by how quickly they are caught, and how long they are locked up for. Auto theft recidivism rates are higher than any other crime, including drug offenses. Around 80% are re-arrested within 3 years of being released. Considering the clearance rate for auto theft is around 10%, that means the average auto thief is stealing a lot of cars before being caught again, and the actual recidivism rate is higher than 80%.
And I'm sure Hyundai and kia stats would be much higher due to lack of immobiliser and visibility of how easy they are to steal on social media.
1. 66,625 vehicle thefts divided by a population of 27,600,000. https://www.abs.gov.au/statistics/people/crime-and-justice/r...
2. 659,880 vehicle thefts divided by a population of 341,784,857 is 193 per 100k. https://www.nicb.org/news/news-releases/us-vehicle-thefts-ex...
Your computer has a built-in list of signatures that it trusts and expects to see on the certificates, and one of those signatures was eMudhra’s. These signatures are basically just a proof that you know a secret number without revealing the number itself. If you know the secret number, you can sign the certificate.
There are an unimaginable number of secret numbers. Imagine each atom in the Great Pyramid of Giza is a secret number. And then imagine you have as many Great Pyramids of Giza as there are atoms in the universe. And then double it.
Now imagine you choose any of the numbers represented by all those atoms to sign the certificate with, and you chose 4.
It’s not technically wrong: 4 is indeed a number. But if anyone wants to guess your number, and they start at 0, it won’t take them very long to find your secret number.
Once they’ve found your secret number, they can now sign certificates pretending to be you. So someone could pretend to be your bank, and present a certificate saying “yes, I am your bank” with a signature on it that your computer trusts.
Yours is much funnier!
https://youtu.be/a6iW-8xPw3k?is=IFYVVkYnlKQk1uh7
In order to sell something in the market, you need to prove that you are who you say you are. There are about 200 trusted clerks whose job it is to check your papers and verify your identity. Once they're satisfied, they stamp your papers with an intricate stamp designed by the finest cryptographers.
Anyone seeing that stamp knows that a trusted clerk has checked your identity and approved your papers. Most people don't examine the stamp closely. They simply recognize it as belonging to one of the trusted clerks and accept it.
One of these clerks, however, instead of commissioning an intricate stamp from a master cryptographer, bought an ordinary rubber stamp from an office supply store.
One day, a particularly paranoid cryptographer was examining the stamps used throughout the market and noticed something peculiar about one of them. It wasn't intricate at all. In fact, it was so simple that anybody with a few dollars and a trip to the office supply store could make an identical stamp.
The clerk's job wasn't merely to stamp papers. The clerk was supposed to check that everything written on them was true before applying the stamp.
But now anyone could write whatever they wanted on their own papers and stamp them. The resulting documents would look exactly like ones that had been carefully checked and approved by the clerk.
A robber could write "thebank.com" on their papers, stamp them with the office supply rubber stamp, and present themselves as thebank.com. If they could get a customer to come to their stall instead of the real bank's, the customer would have no reason to suspect anything was wrong.
After all, the papers had the stamp of a trusted clerk. And everyone who trusted that clerk's stamp would believe them.
Not 100% accurate, but conveys how stupid and incompetent all this is.
https://jeffwerner.ca/kryptonite_lock/
Everyone agrees that passports aren't made by one company or one country, yet all countries accept passports.
Not everyone can just start making passports though, it's a complicated process of gaining trust and validation as a "real country" before you can issue passports.
One "country" started using a bit of paper that said "trust me bro" and everyone had to treat it as a valid passport. Worse, the way the internet works is that the border guards don't actually see who issued the passport, only that one of the trusted countries produced it.
Having passports this easy to forge means anyone can anyone.