“We have a class of products with deterministic cost and stochastic outputs: a built-in unresolved tension. Users insert the coin with certainty, but will be uncertain of whether they'll get back what they expect. This fundamental mismatch between deterministic mental models and probabilistic reality produces frustration — a gap the industry hasn't yet learned to bridge.”
And all the news today around AI being a bubble -
We’re still learning what we can do with these models and how to evaluate them but industry and capitalism forces our hand into building sellable products rapidly
(unrelated) what's the font used for the cursive in the article? the heading is ibm plex serif and the content dm mono, but the cursive font is simply labeled as dm mono which isn't accurate
I will believe this theory if someone shows me that the ratio of scientists to engineers of leading teams of the leading companies deploying AI products is bigger than 1.
> After decades of technical innovation, the world has (rightfully) developed some anti-bodies to tech hype. Mainstream audiences have become naturally skeptical of big claims of “the world is changing”.
Well, it took about 3 years of non-stop AI hype from the industry and press (and constant ignoring of actual experts) until finally the perception seems to have shifted in recognising it as another bubble. So I wouldn't say any lessons were learned. Get ready for the next bubble when the crypto grifters that moved to "AI" will soon move on the to the NEXT-BIG-THING!
This is pure sophistry and the use of formal mathematical notation just adds insult to injury here:
“Think about it: we’ve built a special kind of function F' that for all we know can now accept anything — compose poetry, translate messages, even debug code! — and we expect it to always reply with something reasonable.”
This forms the axiom from which the rest of this article builds its case. At each step further fuzzy reasoning is used. Take this for example:
“Can we solve hallucination? Well, we could train perfect systems to always try to reply correctly, but some questions simply don't have "correct" answers. What even is the "correct" when the question is "should I leave him?".”
Yes of course relationship questions don’t have a “correct” answer. But physics questions do. Code vulnerability questions do. Math questions do. I mean seriously?
The most disturbing part of my tech career has been witnessing the ability that many highly intelligent and accomplished people have to apparently fool themselves with faulty yet complex reasoning. The fact that this article is written in defense of chatbots that ALSO have complex and flawed reasoning just drives home my point. We’re throwing away determinism just like that? I’m not saying future computing won’t be probabilistic but to say that LLMs are probabilistic, so they are the future of computing can only be said by someone with an incredibly strong prior on LLMs.
I’d recommend Baudrillards work on hyperreality. This AI conversation could not be a better example of the loss of meaning. I hope this dark age doesn’t last as long as the last one. I mean just read this conclusion:
“It's ontologically different. We're moving away from deterministic mechanicism, a world of perfect information and perfect knowledge, and walking into one made of emergent unknown behaviors, where instead of planning and engineering we observe and hypothesize.”
I don’t actually think the above paragraph makes any sense, does anyone disagree with me? “Instead of planning we observe and hypothesize”?
That’s called the scientific method. Which is a PRECURSOR to planning and engineering. That’s how we built the technology we have today. I’ll stop now because I need to keep my blood pressure low.
There's another couple of principles underlying the most uses of science, which are consistency and smoothness. That is extrapolation and interpolation makes sense. Also, that if an experiment works now, it will work forever. Critically, the physical world is knowable.
You seem to be having strong emotions about this stuff, so I'm a little nervous that I'm going to get flamed in response, but my best take at a well-intentioned response:
I don't think the author is arguing that all computing is going to become probabilistic. I don't get that message at all - in fact they point out many times that LLMs can't be trusted for problems with definite answers ("if you need to add 1+1 use a calculator"). Their opening paragraph was literally about not blindly trusting LLM output.
> I don’t actually think the above paragraph makes any sense, does anyone disagree with me?
Yes - it makes perfect sense to me. Working with LLMs requires a shift in perspective. There isn't a formal semantics you can use to understand what they are likely to do (unlike programming languages). You really do need to resort to observation and hypothesis testing, which yes, the scientific method is a good philosophy for! Two things can be true.
> the use of formal mathematical notation just adds insult to injury here
I don't get your issue with the use of a function symbol and an arrow. I'm a published mathematician - it seems fine to me? There's clearly no serious mathematics here, it's just an analogy.
> This AI conversation could not be a better example of the loss of meaning.
The "meaningless" sentence you quote after this is perfectly fine to me. It's heavy on philosophy jargon, but that's more a taste thing no? Words like "ontology" aren't that complicated or nonsensical - in this case it just refers to a set of concepts being used for some purpose (like understanding the behaviour of some code).
Building with non deterministic systems isnt new. It does not take a scientist. Though people who have experience with these systems are fewer in number today. You saw the same thing with TCP/IP development where we ended up developing systems that assumed the randomness and made sure that isnt passed on to the next layer. For every game, given the latency involved in previous networks, there is no way on the network games were deterministic.
While this article is a little overenthusiastic for my taste, I think I agree with the general idea of it - and it's always kind of been my pet peeve when it comes to ML. It's a little depressing to think that's probably where the industry is heading. Does anyone feel the same way?
A lot of the stuff the author says resonates deeply, but like, the whole deterministism thing is why I liked programming and computers in the first place. They are complicated but simple; they run on straightforward, man-made rules. As the article says:
> Any good engineer will know how the Internet works: we designed it! We know how packets of data move around, we know how bytes behave, even in uncertain environments like faulty connections.
I've always loved this aspect of it. We humans built the entire system, from protocols down to transistors (and the electronics/physics is so abstracted away it doesn't matter). If one wants to understand or tweak some aspect of it, with enough documentation or reverse engineering, there is nothing stopping you. Everything makes sense.
The author is spot on; every time I've worked with ML it feels more like you're supposed to be a scientist than an engineer, running trials and collecting statistics and tweaking the black box until it works. And I hate that. Props to those who can handle real fields like biology or chemistry, right, but I never wanted to be involved with that kind of stuff. But it seems like that's the direction we're inevitably going.
Even amidst the pressure of "client X needs feature Y yesterday, get it done with maximum tech debt!" it felt like there were still chances to carve out niches of craft, care and quality.
The rise of "probabilistic" software feels like it is a gift to "close enough is good enough" and "ship first, ask questions later".
If only we all became scientists. We'll more likely become closer to tabloid journalists, producing something that sounds just truthy enough to get clicks.
I like this framing, but I don’t think it’s entirely new to LLMs. Humans have been building flexible, multi-purpose tools and using them for things the original inventor or manufacturer didn’t think of since before the invention of the wheel. It’s in our DNA. Our brains have been shaped by a world where that is normal.
The rigidness and near-perfect reliability of computer software is the unusual thing in human history, an outlier we’ve gotten used to.
> Dismissal is a common reaction when witnessing AI’s rate of progress. People struggle to reconcile their world model with what AI can now do, and how.
By probabilistic pattern matching. Next.
> Every model update doesn’t necessarily mean a complete rewrite every time, but it does force you to fundamentally rethink your assumptions each time, making a rewrite a perfectly plausible hypothesis.
Translation: our crap is unreliable and will introduce regressions that you have to manage. It can regress so much that anything you built on this has to be redone. Next.
> I strongly believe that when it comes to AI, something is happening. This time it does feel different.
Said the guy working for the company that is selling you this stuff. Next.
Right now you have this awesome new dynamic capability that doesn’t mesh with how we are used to building software; well defined, constrained, correct. Software products are close ended. Imagine if they weren’t. Imagine if you were playing an open world game like Skyrim or runescape and new areas were created as you explored, new weapons, entirely new game mechanics spontaneously arose as you played. Or imagine an intent based business analytics app, that had a view on your company’s database and when a user wanted a report or a visual it generated it on the fly. We’re only limited by our imagination here.
I tend to agree with most of what has been said. The only difference is that some workflows need to be deterministic otherwise the penalty for failure is high. In that case, AI is helpful in searching through a space, but something orthogonal needs to verify its output.
It seems to me that probabilistic approaches are more akin to magical AI thinking right now, so defending that as the new paradigm sounds quite egregious and reeks of (maybe involuntary?) inevitabilism.
Even if the assumption is correct, forcing a probabilistic system on a strongly deterministic society won't end well. Maybe for society, but mostly for the companies drumming up their probabilistic systems and their investors.
Also, anyone who wants to make money probabilistically is better off going to the casino. Baccarat is a good one. European Roulette also has a better house margin than chatGPT's error margin.
this entire endeavor is a fools errand, and any who has used coding agents for anything more complex than a web tut knows it.
it doesn't matter how much jargon and mathematical notation you layer on top of your black box next token generator, it will still be unreliable and inconsistent because fundamentally the output is an approximation of an answer and has no basis in reality
This is not a limitation you can build around, its a basic limitation of the underlying models.
Bonus points if you are relying on an LLM for orchestration or agentic state, its not going to work, just move on to a problem you can actually solve.
In physics, the change from classical to quantum theory was a change from determinism to probabilistic determinism. There is not one physicist on earth that would ever exhort you to use quantum theory where classical theory will do. Furthermore, when you study physics you must learn the classical theory first or you will be hopelessly lost, just like the author of this article.
The central analogy of the article is entirely bogus.
This article does not rise to the level of being wrong.
What a bunch of pretentious nonsense. It is always a red flag when an author tries to shoehorn mathematical notation into an article that has nothing mathematical about it whatsoever. Gives off "igon value problem"-vibes.
This article can be taken a little less seriously. It is just an opinion/experience of a person.
I think he has a lot of good points there. Yes he treads on the thin ice with the very big statements and generalisations, but those we don't have to "sign under with blood".
I do like the simple formula concept. It does make sense. It's not an ultimate representation of everything, but it's a nice idea of how to frame the differences between the logics we are dealing with.
I choose to not commit the whole message of the article to my core beliefs, but I'll borrow thoughts, ideas for the debates and work ahead.
30 comments
[ 3.8 ms ] story [ 56.7 ms ] thread“We have a class of products with deterministic cost and stochastic outputs: a built-in unresolved tension. Users insert the coin with certainty, but will be uncertain of whether they'll get back what they expect. This fundamental mismatch between deterministic mental models and probabilistic reality produces frustration — a gap the industry hasn't yet learned to bridge.”
And all the news today around AI being a bubble -
We’re still learning what we can do with these models and how to evaluate them but industry and capitalism forces our hand into building sellable products rapidly
https://www.cs.utexas.edu/~eunsol/courses/data/bitter_lesson...
https://news.ycombinator.com/item?id=19393432
Well, it took about 3 years of non-stop AI hype from the industry and press (and constant ignoring of actual experts) until finally the perception seems to have shifted in recognising it as another bubble. So I wouldn't say any lessons were learned. Get ready for the next bubble when the crypto grifters that moved to "AI" will soon move on the to the NEXT-BIG-THING!
“Think about it: we’ve built a special kind of function F' that for all we know can now accept anything — compose poetry, translate messages, even debug code! — and we expect it to always reply with something reasonable.”
This forms the axiom from which the rest of this article builds its case. At each step further fuzzy reasoning is used. Take this for example:
“Can we solve hallucination? Well, we could train perfect systems to always try to reply correctly, but some questions simply don't have "correct" answers. What even is the "correct" when the question is "should I leave him?".”
Yes of course relationship questions don’t have a “correct” answer. But physics questions do. Code vulnerability questions do. Math questions do. I mean seriously?
The most disturbing part of my tech career has been witnessing the ability that many highly intelligent and accomplished people have to apparently fool themselves with faulty yet complex reasoning. The fact that this article is written in defense of chatbots that ALSO have complex and flawed reasoning just drives home my point. We’re throwing away determinism just like that? I’m not saying future computing won’t be probabilistic but to say that LLMs are probabilistic, so they are the future of computing can only be said by someone with an incredibly strong prior on LLMs.
I’d recommend Baudrillards work on hyperreality. This AI conversation could not be a better example of the loss of meaning. I hope this dark age doesn’t last as long as the last one. I mean just read this conclusion:
“It's ontologically different. We're moving away from deterministic mechanicism, a world of perfect information and perfect knowledge, and walking into one made of emergent unknown behaviors, where instead of planning and engineering we observe and hypothesize.”
I don’t actually think the above paragraph makes any sense, does anyone disagree with me? “Instead of planning we observe and hypothesize”?
That’s called the scientific method. Which is a PRECURSOR to planning and engineering. That’s how we built the technology we have today. I’ll stop now because I need to keep my blood pressure low.
I don't think the author is arguing that all computing is going to become probabilistic. I don't get that message at all - in fact they point out many times that LLMs can't be trusted for problems with definite answers ("if you need to add 1+1 use a calculator"). Their opening paragraph was literally about not blindly trusting LLM output.
> I don’t actually think the above paragraph makes any sense, does anyone disagree with me?
Yes - it makes perfect sense to me. Working with LLMs requires a shift in perspective. There isn't a formal semantics you can use to understand what they are likely to do (unlike programming languages). You really do need to resort to observation and hypothesis testing, which yes, the scientific method is a good philosophy for! Two things can be true.
> the use of formal mathematical notation just adds insult to injury here
I don't get your issue with the use of a function symbol and an arrow. I'm a published mathematician - it seems fine to me? There's clearly no serious mathematics here, it's just an analogy.
> This AI conversation could not be a better example of the loss of meaning.
The "meaningless" sentence you quote after this is perfectly fine to me. It's heavy on philosophy jargon, but that's more a taste thing no? Words like "ontology" aren't that complicated or nonsensical - in this case it just refers to a set of concepts being used for some purpose (like understanding the behaviour of some code).
A lot of the stuff the author says resonates deeply, but like, the whole deterministism thing is why I liked programming and computers in the first place. They are complicated but simple; they run on straightforward, man-made rules. As the article says:
> Any good engineer will know how the Internet works: we designed it! We know how packets of data move around, we know how bytes behave, even in uncertain environments like faulty connections.
I've always loved this aspect of it. We humans built the entire system, from protocols down to transistors (and the electronics/physics is so abstracted away it doesn't matter). If one wants to understand or tweak some aspect of it, with enough documentation or reverse engineering, there is nothing stopping you. Everything makes sense.
The author is spot on; every time I've worked with ML it feels more like you're supposed to be a scientist than an engineer, running trials and collecting statistics and tweaking the black box until it works. And I hate that. Props to those who can handle real fields like biology or chemistry, right, but I never wanted to be involved with that kind of stuff. But it seems like that's the direction we're inevitably going.
Even amidst the pressure of "client X needs feature Y yesterday, get it done with maximum tech debt!" it felt like there were still chances to carve out niches of craft, care and quality.
The rise of "probabilistic" software feels like it is a gift to "close enough is good enough" and "ship first, ask questions later".
If only we all became scientists. We'll more likely become closer to tabloid journalists, producing something that sounds just truthy enough to get clicks.
Harmless enough if you are just making images for fun. But probably not an ideal workflow for real work.
The rigidness and near-perfect reliability of computer software is the unusual thing in human history, an outlier we’ve gotten used to.
By probabilistic pattern matching. Next.
> Every model update doesn’t necessarily mean a complete rewrite every time, but it does force you to fundamentally rethink your assumptions each time, making a rewrite a perfectly plausible hypothesis.
Translation: our crap is unreliable and will introduce regressions that you have to manage. It can regress so much that anything you built on this has to be redone. Next.
> I strongly believe that when it comes to AI, something is happening. This time it does feel different.
Said the guy working for the company that is selling you this stuff. Next.
Even if the assumption is correct, forcing a probabilistic system on a strongly deterministic society won't end well. Maybe for society, but mostly for the companies drumming up their probabilistic systems and their investors.
Also, anyone who wants to make money probabilistically is better off going to the casino. Baccarat is a good one. European Roulette also has a better house margin than chatGPT's error margin.
it doesn't matter how much jargon and mathematical notation you layer on top of your black box next token generator, it will still be unreliable and inconsistent because fundamentally the output is an approximation of an answer and has no basis in reality
This is not a limitation you can build around, its a basic limitation of the underlying models.
Bonus points if you are relying on an LLM for orchestration or agentic state, its not going to work, just move on to a problem you can actually solve.
The central analogy of the article is entirely bogus.
This article does not rise to the level of being wrong.
The entire article is a business pseudo-philosophy word salad equivalent of tiktok 'I'm not like other (men/women)' posts.
I think he has a lot of good points there. Yes he treads on the thin ice with the very big statements and generalisations, but those we don't have to "sign under with blood".
I do like the simple formula concept. It does make sense. It's not an ultimate representation of everything, but it's a nice idea of how to frame the differences between the logics we are dealing with.
I choose to not commit the whole message of the article to my core beliefs, but I'll borrow thoughts, ideas for the debates and work ahead.
Ask ChatGPT to fix it.
ps. On a side note, I love that vibe has come back as a word. Feels like the 60s.