To be fair, open source models at this point exceed or are similar to OpenAI's best models. There's really no loss in releasing these weights. It might even have some upsides in terms of attracting talent.
Of course it does. Doesn't make it any less hypocritical for a company called OpenAI to think it's doing some great service to the OSS community by publishing table scraps.
Oh that's interesting, because they distilled only with 1 method, not both methods that were used on R1 (the one was reinforcement learning, the other supervised finetuning, I think only the former was used on the distillations). So there might be some room for improvement. I hadn't seen any third party distillations yet but I'll have a look.
It irks me that I still don't know what an open model means. Or rather, I don't like calling a model that's trained on a closed dataset, with secret techniques as open, even if the weights are publicly available.
10 comments
[ 6.1 ms ] story [ 39.5 ms ] threadOf course it does. Doesn't make it any less hypocritical for a company called OpenAI to think it's doing some great service to the OSS community by publishing table scraps.