128 comments

[ 2.9 ms ] story [ 97.3 ms ] thread
This concept is missed so much in AI research and is quite frustrating.
I wonder how much traffic from bots is skewing OPs nested CSS calculations
That reminds me of an old comic where a guy picks a milk carton from the grocery store shelf and reads in the box: "Now with 0.01% less semen." and he does not know if he's happy or sad about it.
I agree the general premise but do not agree when it comes to browser support.

I feel like we should be building for the 98% or even 95% and force the remaining to upgrade their browsers. I've built for the IE6 - IE11 era for a painful and long time. I do not give a shit if you want to use a 3 year old browser at this point. Go look at a blank screen.

Pragmatically, often users without new browsers and OSses are not the best clients. In ideal world, sure, I want to support everyone. In a world with limited resources, I would better spend my time elsewhere.
If it's uptime it's definitively not much!
1% failure rate of a hundred might be acceptable. 1% failure rate of a million is not.

Isn't that a named law?

The author seems to equivocate by comparing completely different domains.

Whether 98% is acceptable, it depends on the cost of failure, not the percentage itself.

This applies to AI too.

Your classifier might be 98% accurate and it may sound like a lot.

But if it sits inside a car, making thousands of decisions during every trip then you may be in deep trouble.

while true, the people who will read this and then think twice about implementing and applying things are exactly the people who already doing too much thinking
I am not exactly sure what is the article trying to point out
> Truly robust engineering isn’t about what works for most; it’s about gracefully handling the edge cases.

How do you justify this when you factor in cost and time?

I think this single fact is a major source of enshittification in large software products, especially in the era of ML/AI. If your quality is 99%, it sounds like "you have solved your task", but in reality there is a long tail that over time affects nearly every customer.

I've seen this so many times. 99% of search results are good (so within 100 queries you'll hit at least one bad result with p≈0.63), 99% of dashboard panes load normally (so a dashboard with 20 panes is broken in nearly 1 in 5 loads), and so on. If your LLM gets 99% of tool calls right, nearly every session will contain a malformed tool call.

Probabilities are hard for humans, probably.

> Can you imagine a venue refusing entry to former clients 2% of the time just because they’ve “improved their experience”?

This reminds me almost precisely of the dynamics of pro sports in the US and how fans are getting priced out of attending games or even watching teams on TV as organizations shift to bespoke streaming platforms.

It's just mathematical expectation.

Don't look at the simple probability - look at probability * value.

Alternatively, 98% is plenty.

If your business plan requires you to capitalize on more than 98% of the market, it's already a failure. It'll never happen.

As always, it's an "it depends" situation. If your userbase is largely luddites, then maybe you need to support 10+ year old browsers that can't be updated. Otherwise, you can probably just worry about people who are using computers new enough to actually update their browser once a year or better.

The tradeoff is code complexity and engineering time, vs having a larger market. And that's going to be an individual situation for every company.

or if your business plan needs to dedicate 2% of your earnings to litigation from the problems it causes.
It may be a different 2% every time. Eventually you can cover 100% of your users with 2% 50 times, and 100% of your users will feel your software only works 98% of the time. You can get a reputation as an unreliable vendor.

That can happen but it's usually not that extreme. But you should think about it. Negative reputation spreads faster than positive.

And if your software already supports something then all you have to do is not break it. That's usually easier than making it work then first time.

There's no reason code complexity or engineering time has to increase. You can just use the older version features everywhere instead of forking the supported versions.
Browsers have been good enough for pretty much every reasonable purpose for more than 10 years, and compatibility has been really good for that long as well. Is it really challenging or costly to support a feature set that old? In 2017 an app I built worked on Chrome, Safari, Firefox and Edge without any effort or testing other than on Chrome. It still supports all those. At that time, there were probably more than 2% IE users on the general internet but this was B2B.

Just a couple of up-front choices regarding css frameworks and polyfill libraries were all that was required to do that.

I'm open to believing it could be costly for some projects, but I'm more inclined to believe its mostly chasing FOTM, ignorance and laziness that leads to < 98% support.

There's likely always a line somewhere where effort becomes way out of proportion compared to getting that last mile effort.

Arguably, if you only have a website, that won't work for anyone without access to the Internet. So then you should have a physical presence in each of those people's location, and arguably you shouldn't provide any improvements that give me more than physical presence does, so you should not have the website in the first place, since people without the Internet can't use it or you have to keep your website without any improvements over the physical office.

If you only have a website, arguably 2+ billion people currently wouldn't be able to access it.

And it seems odd to bring 150 mil people as an example, when the baseline should be at least 2+ billion with website only.

What about those 30% of audience to update their browser? On our web platform, the team currently displays a message along the lines of: 'Please update your browser; this site relies on features incompatible with your current version'.
Isn't this obvious?

In some categories, certainty and percentages make a lot of difference--surgeries, accidents. In some, they don't--surveys, grades.

It just depends on the category.

This is akin to saying something as obvious as more percentages are more than less percentages.

While I agree with the general sentiment, the problem here isn't developers not being familiar with statistics, it's the simple fact all of this is profit driven most of the time.

I tried to purchase tickets for an event last week. I had to go through Ticketmaster as it was the only official way. They forced me to verify my account using a phone number, but whoever they were using for messages wasn't able to deliver a code to my number. I tried a few numbers from our household and they all failed.

Searching for this issue yielded a bunch of results, so it was definitely a known issue, but there wasn't anything I could do, really. To them, it's simple math. Another SMS provider that covers my (tiny EU) country might be more expensive. They might be avoiding scammers that used my mobile operator in the past. Whatever it is, it would probably cost them more than they lose in ticket sales.

Without some government entity to force them, they don't give a shit about me being able to see an event.

Nice in theory, in practice I remember having to support Internet Explorer about 4 years ago. Hard to justify the investment sometimes, at least polyfills gave use some sanity back. The only reason to do it was: Rich old enterprise customer who can't install chrome due to policies created by Dinosaurs.

Websites are surprisingly hard to maintain long term, specially for a broad audience of devices. Developer Experience can lead to better UX, the easier it is to build/maintain, the more likely we're to do it.

Given how bad AI is at design plus all the unstoppable slop train, I expect websites to become much, much worse.

Covering for the 2% is often not a sound first order business decision. There's certainly higher order benefits. A lot of accessibility features are just plain useful for anyone. And I think companies like Apple generally get that it's a kind of loss leader. But this article makes dishonest TV Shopping Network style arguments like pointing at 150 million as if that was ever an addressable market.

I think either you argue for regulation, or you argue more honestly: asserting that the extra cost will likely never directly pay for itself, but it is some of the secret sauce that can a good product into a great one.