The goal is distillation is to distill into smaller models like 7B, 1.5B. They didn't even change the model size, let alone try a different class of models. Getting expert model's trajectories is trivial if you have…
Wait so Qwen trained QWQ 32B from Qwen 32B and then they distill QWQ back into Qwen 32B? What's the point? This is massive marketing scam here. Borderline academic dishonesty.
The goal is distillation is to distill into smaller models like 7B, 1.5B. They didn't even change the model size, let alone try a different class of models. Getting expert model's trajectories is trivial if you have…
Wait so Qwen trained QWQ 32B from Qwen 32B and then they distill QWQ back into Qwen 32B? What's the point? This is massive marketing scam here. Borderline academic dishonesty.