Using GPT-4 as a probabilistic classifier (twitter.com)

While this is a neat demo, I would in general be cautious about translating GPT-4's logits to probabilities. We know from the technical report that its confidences are not well calibrated. See Figure 8: https://arxiv.org/pdf/2303.08774.pdf

Is this anything new? seems like it was an obvious use case right from the start and just an ad for your product.

Didn't work for me. When I used some negative words negatively, to mean a positive feeling, it did not detect it.