I've seen opus 5 give hallucinated user responses in the middle of a workflow that's supposed to grill a user about their understanding of a topic. Pretty inconvenient to have the model giving out quiz answers as the quiz giver. Other models don't do this. Seems related.
5 comments
[ 1.5 ms ] story [ 38.5 ms ] threadAlso probably related https://news.ycombinator.com/item?id=49118113