tldr: using large expensive models to auto-label data to train small cheap models.
(I find the 'mechanical turk' framing here to be much more confusing & misleading than clever or helpful, and to make it harder to compare to the considerable number of other papers on using language models to generate new datasets & do self-distillation.)
No. Transfer usually means using the same NN model (eg. GPT-3 checkpoints being retrained on Github and then called 'Codex'), or possibly some sort of distillation/sparsifying approach. This is about auto-generating training data, maybe not even meant to be used by a neural net at all.
8 comments
[ 3.0 ms ] story [ 16.1 ms ] thread(I find the 'mechanical turk' framing here to be much more confusing & misleading than clever or helpful, and to make it harder to compare to the considerable number of other papers on using language models to generate new datasets & do self-distillation.)
Naturally. There's a reason AI papers are not published in respected journals a significant proportion of the time.