Call me when a disinterested third-party says so. PR announcements by the very people who have a large stake in our belief in their product are unreliable.
Not to be a luddite, but large language models are fundamentally not meant for tasks of this nature. And listen to this:
> Most notably, it provides confidence levels in its findings, which Cheeseman emphasizes is crucial.
These 'confidence levels' are suspect. You can ask Claude today, "What is your confidence in __" and it will, unsurprisingly, give a 'confidence interval'. I'd like to better understand the system implemented by Cheeseman. Otherwise I find the whole thing, heh, cheesy!
LLMs do typically encode a confidence level in their embeddings, they just never use it when asked. There were multiple papers on this a few years back and they got reasonable results out of it. I think it was in the GPT3.5 era though
14 comments
[ 3.2 ms ] story [ 38.1 ms ] threadTaking CV-filler from 80% to 95% of published academic work is yet another revolutionary breakthrough on the road to superintelligence.
> Most notably, it provides confidence levels in its findings, which Cheeseman emphasizes is crucial.
These 'confidence levels' are suspect. You can ask Claude today, "What is your confidence in __" and it will, unsurprisingly, give a 'confidence interval'. I'd like to better understand the system implemented by Cheeseman. Otherwise I find the whole thing, heh, cheesy!
Funny you say that.
The writing is on the wall that AI will continue to dominate discussion on here for the forseeable future (yawn...)
Its time for me to move to a space where the AI bros are not, wherever that may be. I hope to see some of you there!