HN Explorer

Tell HN: Thanks to thehodge and littlewarden.com, this site is up today

1314 points by dang ↗ HN
A few days ago we got an email from HN user thehodge (https://news.ycombinator.com/user?id=thehodge), aka Dom Hodgson, telling us that HN's SSL cert was about to expire—as indeed it was. All the renewal notices had been going to Scott's old YC email, which no longer works.

Dom runs https://littlewarden.com/, which monitors sites for upcoming issues and lets you know when you're about to publicly embarrass yourself. In a twist on eat-your-own-dog-food (eat someone else's dog food as a service?), he had set up alerts for HN in their system. Lo and behold, it delivered the goods, and that is why you're reading HN as usual today instead of certificate scoldings, and therefore also why my ass is in a saved state, which is how I like it.

I figure the least we can do is proclaim our thanks, so all hail Dom and Little Warden! Yes, I know most of you can do this in 3 lines of Python and a cron job, and yes yes, there are other alert services—but only one has personally helped you waste time unimpeded on the internet. That is all.

80 comments

[ 6.1 ms ] story [ 260 ms ] thread
thehodge
Thanks for the mention Dang, I monitor a few hundred sites of 'importance' and see stuff like this all the time, you are the first one however to thank me for an email saying 'you might want to look into this!'
gavinray
Wow that's really shitty of all the other ones.

Will you name drop them so I can be angry at their ethics for you?

brianpan
Even if my cert is about to expire, I might already know and have a plan to renew and rotate. In fact, I would think MOST large sites have a plan for this and aren't relying on a person from the internet to notice and email them.

Plus, do I need to publicly thank every person that emails and helps me?

remram
There's never a reason to cut it as close as a few days. If you had a sensible plan, you would have enacted it weeks before so as to not get into this situation. Even letsencrypt, with its famously short-lived certificates, is supposed to renew 30 days before expiry.

I have been bitten with this before. I had a good plan, and had actually renewed that one but hadn't reloaded nginx. That's the day I found out that uptimerobot alerts you if your site goes down, but stays willfully silent if it's technically up but using an expired or invalid certificate.

Zababa
Thanks a lot for that!
perryh2
I've thought about building a similar service before and I'm really impressed with all the features you offer, including many that I haven't even thought about before. Many companies use services like Pingdom for uptime monitoring but they don't have nearly as many features. I think you will do really well!
TechBro8615
How many of them become customers? Great strategy!
thehodge
None, tbh I don't try and pitch LW too hard, I just give them a heads up, probably a mistake but feels more ethical :)
cenophor
This comment is just delightful to read.

Thanks for saving us from potential frustrations and for reminding us that helping others can be simply an act of kindness without a monetization scheme attached.

> probably a mistake

I hope you won't ever end up thinking that. :)

rahoulb
As someone who has known Dom for years and started several projects with him, I can say he’s like this all the time and I really doubt he will change.
Abishek_Muthian
Thanks Hodgson, SSL certificate expiring before the times of Let's Encrypt was the stuff for nightmare. I had to dedicate some day every 3 months to renew SSL certificates for my websites.

But now with Lets Encrypt & autocert(Go) it's not the case anymore. But still Little Warden would be useful to detect nasty surprises and besides you're offering other features.

P.S. I've added Little Warden to my curated list of startup tools - https://startuptoolchain.com/#website .

petecooper
https://crt.sh/?q=news.ycombinator.com

(for the curious)

judge2020
Slightly off-topic, but what happened during the time HN was using Cloudflare (August 2017 up to August 2018 by the looks of it)? Was it a trial and enough people complained about the usage, or otherwise had issues accessing from niche user agents?
atok1
I'm not sure with this instance but I do know that people complained since CF is the antithesis of privacy and the free internet.
dang
https://news.ycombinator.com/item?id=21799223
milesvp
I’m surprised by this comment. Fewer moving parts is great, but having managed a news site, where peak traffic can dwarf typical traffic, I’ve just come to accept the necessary evil of a CDN. How are you managing the spikes?
slater
they don't, the site just falls over occasionally :)
Zealotux
Amusing, I embarrassed myself today as I forgot to renew a client's certificate. This kind of service is unfortunately too expensive for my needs (2 small websites to monitor), wouldn't that be possible to have a small software run on my laptop that checks a list of websites every day for upcoming expiration?
geofft
You can do this with the following crappy cronjob (monitoring the machine where your cronjobs run is left as an exercise to the reader / is why you'd want to pay someone to deal with it):

    0 0 * * * openssl s_client -showcerts -connect news.ycombinator.com:443 </dev/null 2>/dev/null | openssl x509 -checkend 864000 >/dev/null || echo "Certificate is expiring"
Assuming your system has local mail (via the sendmail command) working, this will send you an email if your certificate expires in the next 864000 seconds = 10 days. If you have an MTA installed but don't use local mail on the machine, you can use the MAILTO feature to send it to your normal email address.
amozoss
Could pipe it to pushback.io too, super easy way to setup push notifications to your phone
oars
Pushback looks fantastic, thank you for sharing.
atok1
That's pretty useful, thanks.

I can setup a monitor (FOSS) for the computer that is doing the site monitoring, since I only use open source software that I can inspect.

oars
Great one liner to monitor expiring certs, thanks.
cyberge99
dnmin is a small shop that offers it free (I think). I donated the guy $10 for the service a couple of years ago. I got an alert recently, so it works.
amozoss
Google cloud does checks (of endpoints or tcp connections). I've never been charged as far as I can tell. It sends me a text when my site is down, but it has tons of other notification options
dharmab
In addition to monitoring the cert, consider using Let's Encrypt/ACME to auto-rotate certificates.
remram
Unfortunately this also fails in interesting ways...

Just recently, I let one of my certificates expire. The cronjob correctly renewed it, but nginx was not reloaded and kept using the previous certificate. This had never happened before, because I would usually make changes regularly and trigger a reload, which would load the new certificate. Therefore this website had run without issues for 2 years with an incomplete renewal configuration until it finally broke...

dharmab
Yes, we had lots of issues with nginx serving stale configuration, sometimes even after a reload. I learned to distrust nginx's reload and use two or three nginx servers so I could restart one after updating configs.
sundarurfriend
It's surprising how common this is, from big organizations: either letting the certificate expire, or have it be for the wrong domain that clearly belongs to the same org - but most users wouldn't know or care. So it's a good idea for a service, best of luck to thehodge.
thehodge
It happens A LOT, it's mostly because domains, SSLs and other 'tedious' things like that tend to get lost in the business of 'building something billable' and it's easy to thank that a different department owns that bit.
type0
It's even more common when the certificate is issued for longer than 1 year, that's a bad practice- don't do this!
dEnigma
Really cool of you to mention this publicly! As thehodge said in his comments this is far from usual, and most big websites would just silently renew their certificate without giving any credit.

> Yes, I know most of you can do this in 3 lines of Python and a cron job

At first I thought this was a tongue-in-cheek reference to the famous Dropbox dismissal by BrandonM https://news.ycombinator.com/item?id=9224

dang
I actually stole that line from something Dom said in our email conversation...

(Also, this is offtopic but I'm on a quest to get people to realize that BrandonM's comment has been unfairly characterized:

https://news.ycombinator.com/item?id=28293146

https://news.ycombinator.com/item?id=27068148

https://news.ycombinator.com/item?id=23229275)

dEnigma
Funnily enough I just read some more of the context around that comment and have to agree that it wasn't that bad. I think people, including myself, mostly remember the "For a Linux user, you can already build such a system yourself quite trivially by[...]" part. Personally I always find that hilarious because e.g. my mother can trivially use Linux and Dropbox, but I would have an awful time trying to walk her through BrandonM's proposed setup.
stanmancan
I've caught a few of those once it's too late to reply, and I'm not sure I entirely agree with your sentiment.

Unless I'm totally out of touch, I've always seen the comment referenced as either dismissing a simple solution because a complicated one exists, and/or now having a grasp on how complicated a solution is.

  1. For a Linux user, you can already build such a system yourself quite trivially by getting an FTP account, mounting it locally with curlftpfs, and then using SVN or CVS on the mounted filesystem. From Windows or Mac, this FTP account could be accessed through built-in software.
Most recently I referenced the comment in a discussion around Laravel Forge, which deploys and manages servers for hosting websites. I love it personally, but a friend was of the opinion that "it's unnecessary, all you have to do to setup a server is..."

I get that he was trying to be helpful with his post, but that doesn't dismiss the fact that his very first point was "You can already do this by..." and then proceeding to provide a valid, but complicated solution that very few people could do.

There's nothing wrong with his post, but it does act as a good reminder that there's room for products that provide a simple solution where only complicated options exist.

OJFord
Yes I agree, I mentioned it recently and then shortly after saw someone referencing dang's quest in another thread, and felt a bit guilty oh-I-hope-mine-wasn't-taken-that-way.

As I said then: 'Usually when I see it it's used either in a humbling reminder that the future isn't known sense, or a suggestion not to underestimate the value in simplifying, de-nerd-ifying existing things that work.' But also perhaps/hopefully the most 'jerky' ones get flagged out of my view, and unfortunately into dang's.

I certainly don't see (and would flag myself) anything I felt was saying 'Oh this [BM] guy doesn't know what he's talking about, what an idiot, hahahaha'. It's usually just 'oof, isn't hindsight wonderful, I must try to learn from this'.

cedricd
So does that mean that YC is now a paid subscriber to the service? ;).

Very classy callout in any case. I love the story of a startup getting good press for doing something nice. Also this sounds like a really good case study for them to put up.

Poiesis
I bet Dang's public note of thanks is worth so much more than the subscription revenue that it barely matters if they pay or not.
thehodge
Exactly, this post was such a lovely gesture and the HN team know there is an account for them if they want one (gotta say thanks, afterall, we did launch with a SHOW HN post!)
mikewhy
So "we'll pay you with EXPOSURE"?
snet0
I think that comment is mostly pointed at people who don't actually have "exposure" to barter with. I'm sure a lot of people would do commissions for "free" if it meant they got put at the top of BigSite.com.
jtvjan
But, of course, bigsite.coms tend to have the funds required to not have to ask people for free labour.
dmurray
If YC were a paid subscriber, the mail would presumably have gone to the same defunct email address that the cert expiry notice went to.

How does Littlewarden solve that problem? "Personally contacting the face of the site through a back channel" is a great answer, but not so scalable.

thehodge
We allow multiple emails to be notified for issues, as well as the pretty popular Slack integration (along with other messaging services)
account42
None of that really solves the issue of the configured communication channels becoming stale.
[deleted]
(comment deleted)
mutagen
Looks like a great service, and it monitors quite a bit more than SSL / domain expiration.
endisneigh
This is an incredibly wholesome post all around. Wish all internet interactions were like this!
dang
A sadly-now-deleted comment mentioned certdays.sh, which a quick search shows was posted by its author here:

https://news.ycombinator.com/item?id=24967320 (Nov 2020)

Edit: ok, what the hell:

addons.thunderbird.net SSL certificate has expired - https://news.ycombinator.com/item?id=28116117 - Aug 2021 (55 comments)

W3.org Cert Expired - https://news.ycombinator.com/item?id=27363813 - June 2021 (119 comments)

Cmake.org SSL server certificate have expired - https://news.ycombinator.com/item?id=27314666 - May 2021 (3 comments)

.NET NuGet Package Validation Broken: Certificate Expired - https://news.ycombinator.com/item?id=25929235 - Jan 2021 (4 comments)

Krebsonsecurity.com has an expired SSL certificate - https://news.ycombinator.com/item?id=25132182 - Nov 2020 (63 comments)

Intermediate certificate used for issuance of Comodo certs has expired - https://news.ycombinator.com/item?id=23360624 - May 2020 (6 comments)

Techcrunch SSL Cert Expired - https://news.ycombinator.com/item?id=22422227 - Feb 2020 (95 comments)

Microsoft Teams outage due to expired certificate - https://news.ycombinator.com/item?id=22227266 - Feb 2020 (172 comments)

Mozilla Expired Add-Ons Certificate Post-Mortem - https://news.ycombinator.com/item?id=20423221 - July 2019 (3 comments)

DNSCrypt – how expired certificates became a thing of the past - https://news.ycombinator.com/item?id=19830910 - May 2019 (13 comments)

Apidock.com SSL cert is expired - https://news.ycombinator.com/item?id=19731409 - April 2019 (3 comments)

O2 outage due to expired Ericsson certificate - https://news.ycombinator.com/item?id=18622169 - Dec 2018 (89 comments)

Over half the firmwares uploaded to TCSL Armor have invalid certificates - https://news.ycombinator.com/item?id=17993511 - Sept 2018 (17 comments)

All of Oculus’s Rift headsets have stopped working due to an expired certificate - https://news.ycombinator.com/item?id=16541235 - March 2018 (376 comments)

Ask HN: Does YC blog has an expired SSL certificate? - https://news.ycombinator.com/item?id=14580560 - June 2017 (3 comments)

Mac Store Apps Stopped Working Due to Expired Security Certificate - https://news.ycombinator.com/...

thehodge
Yep it happens a lot.. Thanks for linking these, I'll be writing a few of these down for a future blog post ;)
polote
> Yes, I know most of you can do this in 3 lines of Python and a cron job, and yes yes, there are other alert services

Ultimate troll :) Maybe dang is the secret writer of n-gate

temp_praneshp
I hope the writer of n-gate is fine. Nothing since mid-july.
isoprophlex
Every week I still check, and leave slightly saddened, smirking "n-gate continues the war on it's users"

Hope they're okay, and just bored with writing updates.

[deleted]
(comment deleted)
imwillofficial
Awesome story. This has inspired a bit of long forgotten altruism in me.
petercooper
As an aside, I've known thehodge for about fifteen years and he's a total mensch. He ran a popular annual hackathon (an actual ethical one where everyone won a prize) here in the UK, raises money seemingly non-stop for a children's hospice, and has launched perhaps 102 random businesses (including an online candy store!) and side projects over the years both for his own and our entertainment :-D If you want to patronize or support a business that's actually run by a good, ethical person, this is the one for you.
iamben
Agree completely! He's a great guy! Excellent work Dom!
neilwilson
And let's not forget creating "Hodgeland" when the pandemic prevented him taking his little girl to Disneyland.

https://www.bbc.co.uk/news/uk-england-leeds-56099072

Dom is a legend in his own lunchtime. A hugely positive inspiration to everybody who has had the good fortune to meet him.

thehodge
That was 100% my wife, she came up with the idea after our trip to TokyoDisneyland was cancelled and we were all a bit down. I'd love to take credit for that but I cannot!
scrumlord
boo-fucking-hoo

disneyland is for capitalist pigs

onion2k
I met Dom at a few of the Newcastle barcamps a loooong time ago. I'll second that he's an awesome guy, and an impressive serial entrepreneur, and very good at improvised PowerPoint karaoke...
gwintrob
Very kind of you dang and nice site thehodge :)
drallison
Kudos and thanks to Dom Hodgson.
jarofgreen
FYI the little warden menu doesn't work on Firefox or Chrome Android (at least I assume it's meant to be a menu?)

Your various site & DNS checks - do they work on IPv4 and V6? We recently managed to mess up our V6 records whilst our V4 were fine. Turns out our site checkers ran on V4! We've managed to get Prometheus to check on both now (kinda, there are some DNS caching issues somewhere) but now I'm surprised more checkers don't offer this feature.

thehodge
Thanks for the heads up, I'll take a look at that and the good news is that we have a brand new marketing site launching very soon which IS much more mobile compatible!

The IPv6 thing, tbh nobody has tried it yet, but it's certainly something that if it doesn't work, we will fix it so it does :)

jarofgreen
To be clear, I'm suggesting that checkers should run the tests over both IP4 and IP6 to explicitly catch situations where one is set up correctly and one isn't.
scrumlord
Running a website is so hard even a moron can do it.