At 5 cents per neuron with 4o-mini, for pretty satisfying descriptions.
"we fine-tune Llama-3.1-8B-Instruct to directly predict per-token activations ... [this] allows us to use smaller models, and the task of directly predicting the output (integer from 0-10) gets rid of the extra tokens, making the prompt much shorter."
1 comment
[ 0.21 ms ] story [ 15.2 ms ] thread"we fine-tune Llama-3.1-8B-Instruct to directly predict per-token activations ... [this] allows us to use smaller models, and the task of directly predicting the output (integer from 0-10) gets rid of the extra tokens, making the prompt much shorter."