Anecdotal, but working on a biodiversity-related project has made Fable completely useless to me. It sends literally everything to Opus immediately because it’s related to biology. No word from Anthropic since I sent feedback, so I won’t bother relying on it and I don’t plan to.
I get that it’s biology and there are potential risks in that field, but this is stuff like making sure the name of a worm exists in a database or coordinates where the worm was seen are valid. Fable won’t even inspect the purpose, though.
I totally understand the US admin played a significant role here. I don’t think it was the only role, though. I also think the classifier is way too crude. I doubt I could do better on short notice, or perhaps it’s due to external constraints entirely. Considering the cash my organization dumps on them, we hoped for better. Such a disappointment. We’re all pretty leery now and investigating other models.
I actually posted praising Fable’s code review quality when it first debuted, and someone accused me of being an LLM (ha). This new version gets no praise whatsoever, though. It’s such a contrast.
You apply to their biology program at Anthropic and they'll let you in. This is the same advice I gave here, based on a friend at Stanford going through the same thing: https://news.ycombinator.com/item?id=48778446#48781270
Not sure what to tell you then, that's their process and if you don't like it then you won't be able to use their models for biology, it's as simple as that.
LOL. This is not what I was commenting on. Unless their process starts with "never disclose the existence of this program, force the qualifying candidates to stumble upon it by accident"
Most of their programs are invite only, I was merely telling them how they might be able to get on such an invitation list, not that it would be guaranteed. If you know, you know, and if you don't, well, Anthropic doesn't consider your work worthy enough to give access. That is what my friend said it's like when dealing with them for uncensored model usage.
Fable has been a hit or miss for me. It fixed a couple of my Github repos with specialized R packages for uncommon econometrics methods where Opus 4.8 had failed. However, for many apparently mundane requests, it just reverted to Opus.
12 comments
[ 3.3 ms ] story [ 32.8 ms ] threadI get that it’s biology and there are potential risks in that field, but this is stuff like making sure the name of a worm exists in a database or coordinates where the worm was seen are valid. Fable won’t even inspect the purpose, though.
I totally understand the US admin played a significant role here. I don’t think it was the only role, though. I also think the classifier is way too crude. I doubt I could do better on short notice, or perhaps it’s due to external constraints entirely. Considering the cash my organization dumps on them, we hoped for better. Such a disappointment. We’re all pretty leery now and investigating other models.
I actually posted praising Fable’s code review quality when it first debuted, and someone accused me of being an LLM (ha). This new version gets no praise whatsoever, though. It’s such a contrast.
Developing a classifier can be a weird ordeal when you leave the ML world (of making it work) and enter the realm of policy makers applying it.
My guess is they're being ham fisted with little tolerance for false negatives. We can all guess why...
They raised it with Anthropic and were met with silence (as usual), not advice to join the programme.
I hope it's even clearer now.