When did we give up on debugging?
When did we decide to give up on debugging software?
We are creating faulty AI's or at least AI's not provably correct and sticking them in everything from computer mice to industrial machinery, some of which can kill, maim or injure humans without pause. These AI's are given piles of data, a lot of it unchecked which is processed and used to create more data for a next generation of AI and so on.
But everyone seems perfectly content to say "Well, these things happen" when AI's hallucinate. They say "It's too difficult to fix" when it's show than AI's are producing bad results. My favorite: "it's too expensive" when someone would lose profits if they actually tried to fix an issue.
I've been a programmer since the late 80's. I'd like to think that I was taught that, if there is a problem, you fix it. End of discussion. That we write software, which has rules and we follow those rules, and if it doesn't work, we fix it or we (occasionally) make new rules, but follow the same correctness principle.
But the end goal is always to create software that, at the end of the day, does exactly what we tell it to do, the way we tell it to do it. Not "mostly what we tell it to do". Not "mostly the way we tell it to". Exactly. No marketing department equivocation. No "allowances".
We seem to have reached a point that smells like management decided money is more important that correctness and programmers smelled a Porsche in their future and decided to go along with it. Someone decided we could skip a step or two, examined the process and thought "Well, unfortunately we need those programming clowns but I guess we could skip the testers …" and it was good and profitable and all was right with the world, except when it wasn't, but that wasn't profitable, so the world is just hallucinating and oh, we can't fix that … etc.
What. The. F***.
Software is 0's and 1's. There's not a lot of leeway there. It either works and kudos or it doesn't and it's time for some defenestration, in my opinion.
Everyone seems perfectly content to throw money and data at AI and when it screws up, the creators say "Yeah, no one understands the underlying structure anyway, so we just throw more data and money at it, and … oh, hey, Porsche dealer, gotta go! Cheers!" Meanwhile, we're standing in a smoking ruin, wondering where did we go wrong?
Software is either right or it's wrong and if it's wrong, you fire someone (like me) and then you go fix it. Wash, rinse and repeat.
Why are we not demanding this with a gun pointed at a manager's head?
Note: I know this will get downvoted to the depths of Hell for any number of reasons. I'm still asking the question.
7 comments
[ 3.1 ms ] story [ 25.1 ms ] threadThis kind of ranting doesn't belong on HN.
I’ve been doing this since the mid 90s professionally and the mid 80s as a hobbyist.
Can you come up with some real world examples?
We didn't.
> But everyone seems perfectly content to say "Well, these things happen" when AI's hallucinate.
Because that's how they work. They're not designed for perfection; perfection in this realm is impossible.
> Software is 0's and 1's. There's not a lot of leeway there.
There's a ton of leeway there when you get enough 0s and 1s and apply stochastic models.
I suggest you learn more about AI.
Good debugging is possibly the next breakthrough for AI. AI can "90%" achieve many goals, but because it's a black box with its own mind, the remaining "10%" required by humans is almost or beyond 100% the effort of doing everything manually. Good debugging which would make AI less black box, could be the key to human+AI processes that produce something, as high or higher quality than human-alone do, but with less effort.
People (including managers) aren't ignoring hallucinations. Marketers try to gloss over them, but the top AI companies internally work very hard and test their models. Because even from a pure financial perspective, they know that if they keep stalling eventually money will run out, whereas if they achieve something really useful the rewards will be astronomical even compared to what they have now.
We see obvious hallucinations today not because people don't care, but because they're a hard problem to solve. Remember that 5 years ago even generating coherent-sounding responses was hard. Specific hallucinations are obvious, but like how there are specific programs that obviously halt or don't, the general problem is much much harder.
This doesn't engender a lot of confidence in me you have a deeper understanding of how software works.
As far as LLM introspection goes, it's a hot area of research. Just because you don't see any finished products on the market at this time doesn't mean that people stopped caring or that no one does. The Golden Gate Claude experiment shows there's some really interesting parts to delve into.