One cause of "AI" chatbots' accuracy being so poor?
They're trained on so much web content that is plain wrong but authoritative due to apparent endorsement by humans that show tolerance based on understanding that an "AI" next-token predictor lacks.
Example, https://en.wikipedia.org/wiki/Progressive_web_app
"Progressive web apps must use service workers to create programmable content caches."
This statement is factually false. There is no requirement that a PWA do tthis. True is "To create a programmable content cache, a progressive web app would need to use a service worker." But Gemini parrots the false original.
I suspect the reason the false original persists as authoritative is is the human understanding that subconciously translates it to the true version, theteby tolerating it. If queried, they would acknowledge the original is false.
7 comments
[ 0.32 ms ] story [ 7.6 ms ] threadYou see the same pattern across open weight models and homelab setups, as we try to fit the models on devices while preserving capability and getting reasonable throughput
there are lots of things you can measure, designing tasks for evals is an art (which I'm trying to learn)