Why aren't LLM's trained on their own Chain Of Thought?
I realise that without allowing reasoning tokens, a model performs very poorly. It can't perform simple arithmetic or simple logic and hallucinates a bit.
But by allowing it to think a bit and then answer, the result is much better and way more trustable.
This shows a clean RL environment.. or just a nice data-set. Where you prompt the model two times - one without allowing thinking and one with thinking. Penalise the result from non thinking if the result contradicts the answer obtained from thinking.
1 comment
[ 2.1 ms ] story [ 11.3 ms ] thread