LetsEncrypt Outage (letsencrypt.status.io)

I hope no one was migrating infra EOD West Coast

Did the LLM delete this as well?

We're seeing a lot of downstream effects of this at StatusGator. Of course any provider that relies on LetsEncrypt to issue certs (such as Heroku) is affected.

One notable exception is Cloudflare: They famously no longer rely solely on LetsEncrypt.

Shall we have some way of freely encrypting the web that is relying on one authority?

Especially something that needed to be renewed every 90 or is it 40 days now. How about issuing 100 years certificates as a default?

Mostly this should be a non-event due to renewal long before expiration? Although huge deal I suppose for services that require issuing new certifications constantly; Let's Encrypt would be major failure mode for them.

As they move to shorter-lifetime certs (6 days now https://letsencrypt.org/2025/01/16/6-day-and-ip-certs/?utm_s...) this puts it in the realm of possibility that an incident could impact long-running services.

I want DANE!

Good time to note that Buypass offers free certificates over ACME. I have a few of my domains configured to use them instead of LetsEncrypt, just for redundancy and to ensure I have a working non-LE cert source in case LE suffers problems like this over a longer time period.

Example OpenBSD /etc/acme-client.conf:

  authority buypass {
   api url "https://api.buypass.com/acme/directory"
   account key "/etc/acme/buypass-privkey.pem"
   contact "mailto:youremail@example.com"
  }
  domain example.com {
   domain key "/etc/ssl/private/example.com.key"
   domain full chain certificate "/etc/ssl/example.com.pem"
   sign with buypass
  }

I'm sure those six-day lifetime certificates will work out real nice.

Alternatives are available:

https://zerossl.com/ (90 days)

https://www.buypass.com/ (180 days)

I thought I got rate-limited. Bad timing to spin up a new service.

Let's Encrypt stopped its certificate expiration email notification service a while ago, and I hadn't found a replacement yet. As a result, I didn't receive an expiration notice this time and failed to renew my certificate in advance. The certificate expired today, making my website inaccessible. I logged into my VPS to renew it manually, but the process failed every time. I then checked my cloud provider's platform and saw a notification at the top, which made me realize the problem was with the certificate provider. A quick look at Hacker News confirmed it: Let's Encrypt was having an outage. I want to post this news on my website, but I can't, because my site is down due to the expired certificate.

It's DNS, we're working on it. Sorry, thank you for bearing with us.

LetsNotEncrypt. zing!

This is the first time I remember something like this ever happening with LetsEncrypt

Hopefully the thundering herd when service is restored doesn't knock things offline again. I know LE designs for huge throughput (something like 3X total outstanding certificates in 24 hours, at one point) and the automated client recommendations for backoff are pretty good, but there will be a lot of manual applications/renewals I'm sure.

Well, that does it: certificate lifetimes are even shorter now.

If only the same zest applied to probes