I really like this framing. It reminds me of how early steam engines needed governors—a simple mechanical feedback loop—to prevent them from spinning out of control. The engine itself wasn’t "safe" or "stable" by design; stability was something imposed externally through a control mechanism.
In a way, LLMs feel similar. Their internal workings may be probabilistic and unpredictable, but that doesn't mean we can't build external feedback loops—tests, validation layers, human oversight—to steer them toward reliable, useful outcomes. The unpredictability isn’t a flaw; it’s just a raw, unmanaged state that invites control systems around it.
Maybe what unsettles people is that the "chaos" is now at the language layer, where it feels more personal and less abstract than when it's buried in hardware or OS internals. But we've always tamed unpredictable systems with good design—LLMs are just the next place to apply that thinking.
“ but that doesn't mean we can't build external feedback loops—tests, validation layers, human oversight—to steer them toward reliable, useful outcomes”
To what end though, I almost can’t tell if you’re suggesting the difference is, “here’s a recent phd grad without much experience who we can shape into a useful employee” and “this is the bosses son, he’s an incompetent alcoholic, and we can’t fire him, but with enough checks and balances we should be able to keep him from doing to much damage. ( I know you’re saying the first, but culturally it seems like we’re somewhat saying the second. We’re spending billion of dollars on LLMs so you better build the future out of them, useful or not)
And I’m being a bit severe here, but I have regularly had LLM’s suggest they should overwrite my guard rails, like the code comments, even when the comments say “keep this the same.”
So again, great autocomplete that gives me a 30-50% productivity on my work? Yes, will I be writing no code and vibe coding / accept all without reading within the year?
Fair point—and honestly, we’re still figuring that out. Whether LLMs can be reliably "steered" isn’t settled yet because the control systems themselves are still under construction. Part of that process is exactly what’s happening now: users testing the limits, flagging failures, and feeding that back into the loop.
In some cases, the accuracy just isn’t there (Sabrina Hofstadter’s critique comes to mind). But in others—boilerplate writing, brainstorming—the bar is lower, and a little unpredictability is acceptable or even useful.
It’s worth remembering: we didn’t get stable aircraft on the first try either. Early designs were unstable and dangerous until decades of iteration locked in what worked. We may still be in the equivalent of the Wright Brothers era for LLM control systems.
Some of this is a bit myopic on my part. I’m a software engineer, I want to know if I’ll have a job in five years. You’re certainly right on a mountain of writing tasks we have to accomplish that don’t matter that much. Some of this is my own fighting with the experience of vibe coding. I want it to be as good as the hype and when it’s not it frustrates me,
About vibe coding: I'm not feeling it. I've spent too many hours reading code to track down Heisenbugs to believe vibe coders can just wish that away because LLMs take care of such things
I find this both interesting and very wrong. On the one had there seem to be some potential edge cases where this framing could be useful. On the other hand I think “But LLMs are not deterministic” is really code for “I find limited utility in a tool that regularly acts aggressively contrary to my goals”
7 comments
[ 2.3 ms ] story [ 27.9 ms ] threadIn a way, LLMs feel similar. Their internal workings may be probabilistic and unpredictable, but that doesn't mean we can't build external feedback loops—tests, validation layers, human oversight—to steer them toward reliable, useful outcomes. The unpredictability isn’t a flaw; it’s just a raw, unmanaged state that invites control systems around it.
Maybe what unsettles people is that the "chaos" is now at the language layer, where it feels more personal and less abstract than when it's buried in hardware or OS internals. But we've always tamed unpredictable systems with good design—LLMs are just the next place to apply that thinking.
To what end though, I almost can’t tell if you’re suggesting the difference is, “here’s a recent phd grad without much experience who we can shape into a useful employee” and “this is the bosses son, he’s an incompetent alcoholic, and we can’t fire him, but with enough checks and balances we should be able to keep him from doing to much damage. ( I know you’re saying the first, but culturally it seems like we’re somewhat saying the second. We’re spending billion of dollars on LLMs so you better build the future out of them, useful or not)
And I’m being a bit severe here, but I have regularly had LLM’s suggest they should overwrite my guard rails, like the code comments, even when the comments say “keep this the same.”
So again, great autocomplete that gives me a 30-50% productivity on my work? Yes, will I be writing no code and vibe coding / accept all without reading within the year?
In some cases, the accuracy just isn’t there (Sabrina Hofstadter’s critique comes to mind). But in others—boilerplate writing, brainstorming—the bar is lower, and a little unpredictability is acceptable or even useful.
It’s worth remembering: we didn’t get stable aircraft on the first try either. Early designs were unstable and dangerous until decades of iteration locked in what worked. We may still be in the equivalent of the Wright Brothers era for LLM control systems.
The issue is that LLMs cannot explain their reasoning.
LLMs are not expert systems; expert systems provide an answer and explaining their reasoning.