One cause of "AI" chatbots' accuracy being so poor?

1 points by chrisjj ↗ HN
They're trained on so much web content that is plain wrong but authoritative due to apparent endorsement by humans that show tolerance based on understanding that an "AI" next-token predictor lacks.

Example, https://en.wikipedia.org/wiki/Progressive_web_app

"Progressive web apps must use service workers to create programmable content caches."

This statement is factually false. There is no requirement that a PWA do tthis. True is "To create a programmable content cache, a progressive web app would need to use a service worker." But Gemini parrots the false original.

I suspect the reason the false original persists as authoritative is is the human understanding that subconciously translates it to the true version, theteby tolerating it. If queried, they would acknowledge the original is false.

7 comments

[ 0.32 ms ] story [ 7.6 ms ] thread
my hypothesis is they are using flash and quant'd models to keep up with demand in the hardware crunch

You see the same pattern across open weight models and homelab setups, as we try to fit the models on devices while preserving capability and getting reasonable throughput

Trying to preserve something one cannot reliably measure. I wish you luck!
> cannot reliably measure

there are lots of things you can measure, designing tasks for evals is an art (which I'm trying to learn)

[delayed]
don't let perfection be the enemy of good
[delayed]
humans get better at measuring and increasingly reliability across all our technologies, same is happening with llms